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CMMINABOX 

Field of the Invention 

The present invention relates to a method for assisting and expediting an 
5 organization's progression through the levels of the Capability Maturity Model 

(CMM). Specifically, the present invention relates to a method and related system for 
arranging and administering an organization's infrastructure and a project of interest 
so that the organization and the product may be more mature, as measured by the 
CMM. 

10 

Background of the Invention 
The CapabiUty Maturity Model® (CMM®) may refer specifically to the 
Capability Maturity Model for Software (SW-CMM) or, more generally, to a number 
of other process improvement models developed by the Software Engineering 

1 5 Institute (SEI) and registered to Carnegie Mellon University. The SW-CMM was the 
first model developed by the SEI, and it originally evolved from the need for the 
United States Department of Defense to have another measure besides "lowest 
bidder" in determining who should win project bids. Specifically, the Department of 
Defense desired a method to better compare and distinguish well designed and 

20 shoddy, defective products. The two major usages of the SW-CMM are: (1) as a 

model for Software Process Improvement (SPI) and (2) as a measure of the capability 
to produce quality systems. Specifically, the CMM may help a purchaser differentiate 
properly working product from an incomplete, nonfunctioning, poorly designed 
product by providing information on a producing organization and its production and 

25 development procedures. 

The CMM is an example of a model-based improvement approach that 
focuses on creation process quality. The rationale for this focus is that, unlike 
hardware, manufacturing software is essentially error free (i.e., the production of the 
disks containing the program), but the quality defects (i.e., bugs) are produced during 

30 the concept and development process. Therefore, waiting to identify defects after 

creation of the product is generally difficult and costly. The CMM may be used as a 
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guideline for prioritizing limited resources on the most important, foundational 
improvements. In the SW-CMM, Key Process Areas (KPAs) define "building 
blocks" based on industry best practices. The ultimate goal is to establish "continual 
improvement" of the software engineering process and the resulting products, kaizen 
(Statistical Process Control), which is common in nonsoftware engineering 
disciplines. The CMM is described more fully in Mark C. Paulk, The Capability 
Maturity Model: Guide lines for Improving the Software Process (The SEI Series) 
(Addison- Wesley Pub Co.) (1995). 

The Capability Maturity Model Integration^ (CMMP®) was developed to 
integrate the SW-CMM and various other existing models into a common model. The 
developers of the CMMI are seeking to establish common terminology between the 
models, as well as identifying commonality and variability. The SEI is expected to 
release Version 1 . 1 of CMMI in the near future. 

The SW-CMM model defines five capability levels and identifies Key Process 
Areas (KPAs). The CMMI model replaces the KPAs with Process Areas (PAs). The 
lower levels of the CMMI and the related PAs focus mainly on management 
processes and industry minimal standards. Higher CMMI levels and the related PAs 
generally focus more on organizational and technical processes. The higher levels 
and their PAs also strive for "industry-best" practice. 

While the entire scope of the CMMI is vast and generally outside the range of 
this document, the various levels of the CMMI are now quickly described. At level 0 
or "Incomplete", a project has not yet started. Upon initiation and existence of the 
project, the project is at level 1. At "hiitial" or level 1, the product conditions are ad 
hoc, chaotic, and high-risk. At "Repeatable" or level 2, the project may repeatedly 
perform some functions with difficulty. Relevant PAs to level 2 are Requirements 
Management (RM); Project Planning (PP); Project Monitoring and Control (PMC or 
PC); SuppUer Agreement Management (SAM or SM); Process and Product Quahty 
Assurance (PPQA or QA); Configuration Management (CM); and Measurement and 
Analysis (MA). 

At "Organizationally Defined" or level 3, the relevant PAs include 
Requirements Development (RD); Technical Solution (TS); Product hitegration (PI); 
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Validation (V a); Verification (Ve); Organization Process Focus (OPF or PF); 
Organizational Process Definition (OPD or PD); Organizational Training (OT); 
Integrated Project Management (IPM or IM); Risk Management (RSKM or Rk); 
Decision Analysis and Resolution (DAR or DA); Organizational Environment for 
Integration (01); and Integrated Teaming (IT). 

At "Quantitatively Managed" or level 4, the relevant PAs are Quantitative 
Process Management (QPM or QM) and Organizational Process Performance (OPP or 
OP). QPM relates to the informed and correct use of rigorous statistical techniques 
such as statistical process control (SPC), with the focus on removing specific or 
attributable causes of variance, and OPP relates to the use of statistical techniques to 
measure process efficiency. The fifth and highest level, "Optimizing", is basically 
equivalent to bottom-up process improvement or continuous improvement. In CMMI, 
the level 5 PAs are Organizational Innovation and Deployment (ODD or ID) and 
Causal Analysis and Resolution (CAR or CA). 

The CapabiKty Maturity Model for Software (S W-CMM) was the first, but not 
the only, model for improvement of software development. Some other models 
developed by the SEI include: Integrated Product Development CMM (IPD-CMM), 
which was renamed and incorporated into CMMI Integrated Product and Process 
Development (IPPD); People CMM (P-CMM) for Training, Career Development, and 
Human Resource-related issues; Personal Software Process"" (PSP"™); Software 
Acquisition CMM® (SA-CMM); and Systems Engmeering CMM® (SE-CMM), 
which is being incorporated into CMMI for Systems Engineering/Software 
Engmeering. Similarly, FAA-iCMM (a model similar to CMMI and incorporating 
elements of SW-CMM, SE-CMM, and SA-CMM) was developed by the Federal 
Aviation Administration. 

Achieving higher levels of CMM maturity is a desu-able goal in itself because 
it generally implies that an organization is producmg a superior product and services 
since the higher levels of the CMM generally require the existence of infi-astmcture 
and procedures leading to better tested and developed software and other products. 
As suggested above, organizations also have secondary fmancial incentives to achieve 
higher CMM levels, because customers, such as the United States Department of 
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Defense, are increasingly requiring software suppliers to have a sufficiently high 
CMM level (e.g., at least level 2) before being awarded a contract. 

A threshold problem for many organizations is that the requirements for the 
different maturity levels are relatively complex to understand and implement. It is, 
therefore, a goal of the present invention to provide a method allowing businesses to 
achieve higher CMM levels without having to understand the complicated 
requirements of each level. 

Furthermore, the process of achieving higher CMM levels of increased 
maturity is typically a difficult, expensive, and time-intensive process. While some of 
the costs are unavoidable, many of the difficulties of achieving higher CMM levels 
occur because the requirements for the levels do not fit well within the general 
operations and structure of most organizations. Drastically changing an 
organization's structure and operations is generally a complex and difficult process. 
Therefore, another goal of the present invention is to provide a method that simplifies 
and potentially accelerates the process of modifying an organization's operations and 
structure to meet the requirements of the higher CMM levels. 

Similarly, many organizations also have difficulty implementing changes to 
achieve higher CMM or CMMI levels because the organization use of these maturity 
models as merely checklists of criteria. The maturity models, while serving as a 
measure to assess organizations, offer little guidance to organizations on 
implementation of the specified criteria. The random implementation of the items on 
a maturity model checkUst results in increased time and cost for maturation in 
comparison to carrying out systemic changes that may concurrently satisfy multiple 
checklist items and assist the organization in achieving several checkhst items. 
Furthermore, a piecemeal implementation of the CMM worsens the above-described 
problems of complexity and cost. It is, therefore, another goal of the present 
invention to provide a method by which organizations may implement systemic 
changes to achieve higher levels of CMM maturity. 



Summarv of the Invention 
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In response to these and other needs, the present invention provides a method 
and related system for assisting and expediting an organization's transformation 
toward higher levels of the Capability Maturity Model (CMM) or other derivative 
maturity models, hi particular, the present invention provides a method for producing 
a more mature product. A preferred embodiment of the method comprises the 
managing an organization developing the product, whereby the organizational 
management comprises managing personnel of the organization and implementing a 
product improvement process. The method may further comprise managing a project 
for developmg the product and managing the delivery of the product. Furthermore, 
actions imdertaken during the organizational management affects implementation of 
the project and delivery managements, and the actions undertaken during the project 
and delivery managements Hkewise affect implementation of the organizational 
management. In another embodiment, this method may be implemented using a 
combination of both electronic hardware and software. 



Brief Description of the Drawings 

A more complete xmderstanding of the present invention and advantages 
thereof may be acquired by referring to the following description taken in conjunction 
with the accompanying drawings, in which hke reference niunbers indicate like 
features, and wherein: 

FIG. 1 is a flowchart depicting the steps in a method for producing more 
mature products in accordance with an embodiment of the present invention; 

FIGS. 2A-2J are flowcharts depicting the steps of the process stage of 
organization management in accordance with embodiments of the method of FIG. 1; 

FIGS. 3A-3D are flowcharts depicting the steps of the personnel stage of 
organization management in accordance with embodiments of the method of FIG. 1; 

FIGS. 4A-4F are flowcharts depicting the steps of program management in 
accordance with embodiments of the method of FIG. 1 ; 

FIGS. 5A-50 axe flowcharts depicting the steps of project management in 
accordance with embodiments of the method of FIG. 1; 
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FIGS. 6A-6B are flowcharts depicting the steps of delivery management in 
accordance with embodiments of the method of FIG. 1; 

FIGS. 7A-7E are flowcharts depicting the steps of analysis stage of the 
delivery management of FIG. 6A in accordance with embodiments of the method of 
FIG. 1; 

FIGS. 8A-8J are flowcharts depicting the steps of design stage of the delivery 
management of FIG. 6 A in accordance with embodiments of the method of FIG. 1; 

FIGS. 9A-9M are flowcharts depicting the steps of build and test stage of the 
delivery management of FIG. 6A in accordance with embodiments of the method of 
FIG. 1; 

FIGS. lOA-lOF are flowcharts depicting the steps of deployment stage of the 
delivery management of FIG. 6A in accordance with embodiments of the method of 
FIG. l;and 

FIGS. 1 1 A-B depicts systems for implementing the method of FIGS 1-lOF in 
accordance with various embodiments of the present invention. 

Detailed Description of the Preferred Embodiment 
As generally illustrated in FIG. 1, the present invention provides a CMM in a 
BOX method 10 for easing and speeding an organization's transformation toward 
higher levels of the above-described CMM hierarchy. The CMM in a BOX method 
10 generally comprises the steps of getting started 20, organization management 100, 
program management 400, project management 500, and delivery management 600. 
As suggested in FIG. 1, the CMM in a BOX method 10 performs as a cycle in which 
actions performed during the organization management 100 help conti-ol the current 
steps of program management 400, project management 500, and delivery 
management 600. Subsequently, the actions performed during program management 
400, project management 500, and delivery management 600 adjust the step of 
organization management 100. Each of these steps of CMM in a Box method 10 is 
described in greater detail below. 

In these discussions, it should be appreciated that the various steps of the 
CMM in a Box method 10 preferably include the creation or updating of various 
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documentation (or monuments) that detail and verify the execution of tasks performed 
by the organization. These documents may be used to demonstrate compliance with 
the higher levels of the CMM or CMMI. Some of these documents are listed directly 
with the associated steps, but a complete listing is beyond the scope of the present 
application. A short listing and summary of some of the various documents that may 
be created or updated during the steps of the CMM in a Box method 10 is attached 
hereto as Appendix A. 

The CMM in a BOX method 10 begins with getting started step 20. In step 
20, the organization prepares to initiate the other steps in the CMM in a BOX method 
10. hi particular, the organization may review the requirements of the various 
management steps 100, 300, 400, and 600. Similarly, the organization may review 
the CMM or CMMI and their general requirements in order to better imderstand the 
goals to be accomplished during the various steps of the CMM in a Box method 10. 

Organization Management 

As illustrated in FIG. 1, Organizational Management 100 is divided into two 
stages, process step 200 and personnel step 300. The Organization management 
step 100 generally concerns activities related to the structure and activities of an 
organization. The process stage 200 contains the methodologies, process flows, tools, 
and templates to create and maintain a Software Engineering Process Group (SEPG). 
It should be noted that in the CMMI, the SEPG is replaced by a Process Group to 
allow for the inclusion of systems engineering. Thus, this application uses the SEPG 
to refer to a group overseeing software and non-software processes. As suggested by 
its title, the personnel stage 300 contains the methodologies, process flows, tools and 
templates to perform organizational design and development, measurement 
performance, and conduct organizational training. 

As depicted in FIG. 2A, the process stage 200 consists of the steps of planning 
and organizing a SEPG, step 201; and of managing and improving the organization's 
processes, step 202. Step 201 is further subdivided into planning SEPG project 
execution (step 210) and organizing SEPG project resources (step 220). Likewise, 
managing and improving the organization's processes in step 202 may be subdivided 
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into controlling SEPG project work (step 230); rolling out and supporting SEPG 
projects (step 240), completing the SEPG project 290, and controlling process 
improvement (step 203). In turn, the step of controlling process improvement, 
step 203, consists of conducting a super SQA review, step 250; conducting 
assessments, step 260; conducting quarterly surveys, step 270; and conducting process 
improvements, step 280. 

In the planning and organizing of the SEPG in step 201, the organization first 
performs the planning of the SEPG project execution, step 210. While planning 
SEPG project execution in step 210, the SEPG defines its process improvement plan 
and subordinate plans for the fiscal year. Since the SEPG is a continuously operating 
project, plans are reviewed and updated annually, at a minimum, usually with the 
beginning of a new fiscal year. Step 210 begins at the initiation of the project to 
define the pieces of an initial project plan and all subordinate plans that should be 
used to manage the execution of the project. Using this information, the organization 
seeks to develop a SEPG project plan, a SEPG work plan, a communication and 
sponsorship plan, a configuration management plan, a risk management plan, and a 
training needs matrix, as these objects are defined in the CMM. The organization 
further performs decision analysis and resolution during the planning of the SEPG 
project execution, step 210. 

One possible process for planning the SEPG project execution, step 210, is 
generally depicted in FIG. 2B. In an initial aspect of the planning a SEPG project 
execution, step 210, the organization tailors the CMM in a BOX method 10 as 
needed. Specifically in step 212, the organization determines whether to waive or 
skip steps in the CMM in a BOX method 10 as required by organization or the 
particular project. For instance, the organization skip tasks that are inapplicable to a 
project and therefore unneeded to either achieving higher levels of maturity in the 
CMM or to develop more mature products. 

Another step in the SEPG project execution, step 210, is to develop a project 
plan, step 214. The project plan describes the project approach for the project 
timetable, metrics, organization, supplier agreement management, communication and 
sponsorship strategy, training, quality initiatives, software system development 
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process, configuration management, logistics, facilities, tools, and purchasing. It 
further describes the project approach for training, metrics tracking, and roles and 
responsibilities on the project. The organization may also use Decision Analysis and 
Resolution (DAR) to develop the Project Plan, as defined in the CMMI. 

The organization may further develop subordinate plans, step 216. The 
development of the appropriate subordinate plans, step 216, satisfies the needs of the 
project, such as the creation of subordinate plans for subcontractor management, risk 
management, communication and sponsorship, and configuration management, all of 
which are described in greater detail below. In the development of subordinate plans, 
step 216, the organization may further create a work plan. For instance, the 
organization may create a "bottom-up" or task-level project work plan based upon 
estimates where critical paths and dependencies are defined and managed within a 
project work-planning tool, such as Microsoft Project and Project Workbench®. 

Another aspect of the SEPG project execution process, step 210, is to develop 
project estimates, step 218. The organization may develop project estimates, 
step 218, using an estimating tool as a starting point for the estimates. For instance, 
estimates may be developed using the following steps: (1) tailor tasks and estimating 
model; (2) determine estimating factor values; (3) define work packages; 
(4) determine a timeline for the estimate; (5) reconcile a present estimate to an initial 
estimate; and (6) document assumptions used to form the estimates. The organization 
preferably further validates any estimates by verifying estimates against estimates or 
actual results from comparable projects. To form accurate estimates of available 
resources, the organization should further consider other resource-tapping activities 
such as community involvement, recruiting, mentoring, and training, when evaluating 
resources. 

Returning to FIG. 2A, the organization then continues the process stage 200 
and the planning and organizing the SEPG, step 201, by organizing the SEPG project 
resources, step 220. During step 220, the SEPG focuses on obtaining, assignmg and 
training its human resources, and estabhshing the project's other physical resources 
including installation of tracking tools and document repositories. This task is 
performed iteratively as needed to organize, mobilize and manage SEPG resources 
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throughout the execution of the project. The organization performs step 220 as 
needed to organize the project's human resources, to establish other resources, to 
make work assignments and to any training needed to enable resources. Turning to 
FIG. 2C, the first step in organizing the SEPG project resources in step 220 is to 
refine resource needs, step 221. In this step 221, the organization defines the team 
organization structure, schedules the work, and defines the human and physical 
resource needs of the project. These tasks are performed in view of each project's 
requirements. By refining resource needs in step 221, the organization helps to ensure 
that project staffing and facilities needs are met on a timely basis without affecting the 
completion date and the quality of the work. The organization may complete this 
refining of resource needs in step 221 by: (1) determining project organization 
structure; (2) balancing a development schedule using hxrnian resource guidelines; and 
(3) refinmg physical resource needs that were outlined in the logistics, facilities, and 
tools section of the project plan formed in step 214. 

Returning to FIG. 2C, the organization continues the organization of the SEPG 
process resources in step 220 by establishing project standards and goals, step 222. 
The establishment of project standards and goals in step 222 is accomplished by 
developing, modifying, and adopting administrative and project-specific project 
standards and procedures. Examples of administrative procedures are employee 
availability checklists, time accounting procedures, status reporting, vacation 
scheduling, etc. Project standards and procedures include design and development 
standards, and the use of project specific tools. 

The organization continues the organizing the SEPG process resources in 
step 220 through organizing a project team in step 223, also illustrated in FIG. 2C. 
The selection of project team members is based on project requirements. Other 
elements in the organization of a project team are the finalization of the project team's 
organization structure and documentation in an organization chart in the project plan. 
The organization should fiirther update the training needs matrix to document: (1) the 
training required of each project team member and (2) the proposed means for 
fulfiUing the traming. The training needs matrix is further used to track project team 
member training. In another implementation, organizing a project team in step 223 
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may further require the organization to determine, as a team, the project's mission, 
vision, and charter, and then to document these determinations in the project plan and 
orientation binder that are created as required to achieve higher maturity levels in the 
CMM. 

Returning to FIG. 2C, another task in the organization of SEPG project 
resources is to establish other resources indirectly needed for the SEPG project, 
step 224. Specifically, the organization performs this task by organizing the physical 
resources, such as hardware or software, provided by program management and 
developing the orientation and/or training needed to support the activities of the 
project team. The establishment of other resources in step 224 helps create a work 
environment that promotes communication, collaboration, and group cohesion. 

Also, as illustrated in FIG. 2C, the organization of SEPG project resources in 
process 220 further includes enabling resources, step 225. An organization performs 
this step 225 to orient and train team members, to coach and evaluate team members, 
and to manage the physical resources assigned to the project. The enabling of 
resources in step 225 aids the project manager in motivating and challenging team 
members, while helping to ensure that various project personnel believe their work to 
be important. Specifically, the organization should communicate the project's 
mission, vision, and chater to new team members. Large projects may also elect to 
formalize these items at the program level, and projects may conduct one or more 
meetings that include all team workers. 

Referring to FIG. 2A, another element in the process stage 200 is to manage 
and improve the organization's processes, step 202. The first step in the management 
and improvement of process is the control of SEPG project work in step 230. During 
the control of SEPG project work in the step 230, SEPG project management 
monitors the execution of the project against project plan and makes adjustments as 
necessary. Project Status Reports are prepared for the Project Sponsor. Potential and 
actual problems are identified through the measuring and monitoring of progress and 
performance against the SEPG Project Plan. Depending on the type of problem 
identified, an Issue, Risk, System Investigation Request (SIR) or Change Request 
(CR) is logged. The SIRs and the CRs are described in greater detail below. SEPG 
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Project management is expected to take appropriate corrective actions to resolve 
problems that are discovered. The controlling of SEPG project work in the step 230 is 
also illustrated in FIG. 2D and is now described in greater detail. The controlling of 
SEPG project work in the step 230 includes releasing work packages, step 23 1 . Work 
packages are generally described in the CMM criteria and generally relate to the tasks 
and functions given to the various workers in a project. To release work packages, 
the organization should (1) assemble and release work packages according to the 
work plan and (2) communicate the requirements of the work packages to the 
assigned team members. The project team then performs the work needed to develop 
the required deliverable good. During step 231, the organization preferably acts to 
ensure that each team member understands assigned responsibilities, including target 
dates and budgets. Furthermore, the organization should implement the project so 
that each team member (1) is able to provide input regarding various responsibilities 
and (2) accepts these responsibilities. 

As depicted m FIG. 2D, a following task in the control of SEPG project work, 
step 230, is measuring performance, step 232. The task of measuring performance in 
step 232 generally includes capturing actual results and calculation of metrics in order 
to manage performance. The capture metrics are outlined in the SEPG project plan 
formed in step 214 and include cost, effort, scope, quahty, and schedule. The 
organization should further track project infrastructure and technical requirements, 
such as hardware, software, and performance requirements that were outlined during 
planning in step 210. The orgmization should also analyze any deviations from the 
project plan and identify, in a timely manner, the causes for the deviations. 

Concurrent with measuring of performance in the step 232 is managing 
performance, step 233, as illustrated in FIG. 2D. Managing performance in step 233 
generally requires the organization to manage project performance against the 
previously defined project and work plans. To manage project perform^ce in view 
of the project and work plans, the organization proactively assesses performance, 
status, quality and risk. When the actual results from the development of the project 
do not match the plans, the organization should further determine alternative goals or 
actions. The implementing organization may further obtain approval for corrective 
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actions, and then take corrective actions. The corrective actions may include, but are 
not limited to, work process changes, team building, training, increased or decreased 
supervision, work assignment changes, reassignment of team members, initiation of 
risk responses, the change of requests to be pursued with program management as part 
of the configuration management process, project replanning changes that specify 
needed modifications to the project plan, project plan revisions (work package 
changes, etc.) or escalation to program management. The organization should also 
reevaluate project decisions throughout the project life cycle, when various project 
triggers or other issues, risks, etc. arise. The organization may also manage team 
member performance according to organizational and industry standards and tools. 

Continuing with FIG. 2D, following the measuring of performance in step 232 
and the managing of performance in step 233, the organization communicates project 
status, step 234. During step 234, the organization generally develops and 
communicates project status to all project stakeholders according to the project plan. 
The project stakeholders include project and senior management and other affected 
groups. The organization may further conduct status and review meetings involving 
affected groups as appropriate. During the communication of project status in 
step 234, the organization should document meeting minutes as required for the 
CIVlM. 

Continuing with FIG. 2D, following the communication of project status in 
step 234, the organization obtains acceptance of interim deliverable goods, step 235. 
Obtaining acceptance of interim deliverable goods in step 235 generally requires that 
the organization obtain acceptance of interim deliverables by all designated 
stakeholders, as appropriate, at key interim points throughout the project life cycle. 
Any acceptance of final deliverables takes place in connection with completing the 
program. 

Concurrent with the above-described steps 231-235, another task in the control 
of SEPG project work in step 230 is to execute project management processes, 
step 236. The organization should execute step 236 in conjunction with other project 
control activities, such as measurement activities and status reporting. Also, the 
project management processes may occur continuously, periodically, or may be event 
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driven. One project management process in step 236 is risk management, which 
addresses the identification, analysis, and avoidance/mitigation aspects of risk 
management on a project. During risk management, the organization may perform 
risk identification, during which the organization identifies, names, and describes the 
5 various risks. The organization should further generate a list of specific incremental 

risks in the project's risk tracking tool. For instance, the organization may document 
known triggers for a risk, the potential damage for each risk item, and references for 
the sources of risk. Another risk management task in step 236 is risk analysis, in 
which the organization analyzes the identified risks, hi the risk analysis, the 

1 0 organization should classify the risks and include any additional information 

necessary to support the analysis. The organization may then select a rank/prioritized 
Ust of top risks. For instance, the organization may create a list of the top five risks to 
a project. Another risk management task is risk avoidance and mitigation. Risk 
avoidance activities address the sources of a risk, thereby reducing the probability that 

15 it would become a problem. For a top ranked or prioritized risk, the organization 

should identify how the risk can be avoided. Risk mitigation measures attack the 
consequences of a risk, reducing the risk's potential impact on the project. For the top 
ranked/prioritized risks, the organization may identify actions to reduce the impact of 
the risk if it occurs. The organization may also use Decision Analysis and Resolution 

20 (DAR) to assess the risks, where DAR is defined above. Many automated risk 

management applications are commonly available, and an organization may choose 
fi-om these various risk management appUcations as needed to best fiilfill the needs of 
the organization. 

Another task in the execution of project management in step 236 is scope 
25 management, which addresses the acceptance of requirements to define scope and the 

requirements to change control process. For instance, one scope management task is 
requirements development. During the task of requirements development, the 
organization identifies and documents requirements needed to promote and ensure 
bidirectional traceability, so that the organization may trace requirements between the 
30 development and the testing of the requirements. As with all work products, 

requirements are preferably placed under configuration management (CM), as defined 
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in the CMMI. Another scope management task is requirements acceptance, during 
which the organization documents and reviews requirements with all affected groups 
and obtains acceptance from the affected stakeholders. The organization should 
further establish baseline standards for satisfying the requirements. Another scope 
management task for the organization is making any required changes to the 
requirements and their baseHnes. The organization generally follows the project's 
change control process for any changes to baselined requirements. Namely, the 
organization submits a change request; reviews a change request; performs impact 
analysis, including cost, schedule and efforts impacts; determines disposition; 
implements change, including associated impact to other work products and activities; 
and notifies requester and affected groups. Again, the organization may determine if 
it is necessary to use DAR to assess changes in scope. 

Another project management process in step 236 in the execution of the 
project management processes is configuration management. This task addresses the 
set of activities performed to establish and maintain the integrity of the project work 
products throughout the project's life cycle. One set of configuration management 
tasks relates to configuration identification activities. During the configuration 
identification activities, the organization identifies, names, and describes each of the 
configuration items that should be placed under configuration management. In 
particvdar, all work products should be placed imder some type of configuration 
management. During the configuration identification activities, the organization 
generally uses the CM plan to define a baseline for the configuration items and to 
indicate the level of configuration management for each item. 

Another configuration management process in step 236 is the configuration of 
control activities. Generally, the organization requests, evaluates, approves or 
disapproves, and implements changes to the baselined configuration items defined 
during the configuration identification activities. All of the configuration items 
should be archived and placed under the project's documented change control 
process. 

Configuration of status accounting activities is another configiaration 
management process in step 236. During this process, the organization records and 
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reports the status of the project's configuration items. Similarly, the organization 
should further perform configuration audits. Specifically, the organization may, using 
the CM plan, determine the extent to which actual configuration items reflect the 
planned configuration items. The purpose of this task is to ensure that the entire 
configuration is correct and complete. The organization should fiirther document 
results as required in the CMMI. 

Another project management process of the execution of the project 
management process in step 236 is issue management and escalation. This task 
involves the identification and documentation of issues using an issue tracking tool, as 
well as a review of the issue and an analysis of any impact on deliverables, scope, 
contingency, resources, costs, schedule, and/or quality. Specifically, the organization 
should identify a resolution approval party, an issue's owner, and determine expected 
time fi-ames. The organization may also determine if it is necessary to use DAR to 
assess the issue, as described above. The organization may further research and 
identify issue solution alternatives. Subsequently, the organization may refer the 
issue to program/senior management when: (1) the project cannot resolve the issue 
internally, (2) when the issue impedes the progress of a project, and when the issue is 
beyond the authority of the project manager to resolve. These are generally issues 
that: (1) cannot be resolved within a project team, (2) are resolvable with action 
items, (3) can be escalated to the next level, (4) Are reactively discovered during the 
course of development, (5) affect program/project scope, costs, schedule, projected 
business performance, or high level design, (6) affect multiple projects or releases, 
and/or (7) involve groups outside the project that affect project delivery. The 
organization should accordingly monitor issues status and approve or reject 
resolutions. At the same time, the organization should communicate resolutions to 
stakeholders and affected parties and take corrective action as described above in the 
context related to management of performance tasks. 

Returning to FIG. 2D, another step during process of controlling the SEPG 
project work in step 230 is updating the project plan and subordinate plans, step 237. 
In particular, throughout the life cycle of the project, the project plan and subordinate 
plans (Risk Management, Configuration Management, Work Plan, Subcontractor 
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Management Plan, Community and Sponsorship Plan) should be updated as 
appropriate by the organization to reflect any changes on the project that would effect 
the content of the documentation. 

Referring again back to FIG. 2A, another task of the management and 
5 improve process 202 in project stage 200 is the rollout and support of SEPG projects, 

step 240. During the rollout and support of SEPG projects in step 240, new projects 
to be supported by the SEPG are identified and SEPG processes and tools are 
delivered to them. SEPG Liaisons may conduct process reviews of the SEPG- 
supported projects. Other project-created items referenced during the rollout and 

10 support task include the Service Level Agreement, Tailoring & Waiver Request, 

Metrics Workbook and Metrics Plan. The organization performs this task of step 240 
to rollout SEPG processes and tools throughout the organization. The process of 
rollout and support of SEPG projects in step 240 is illustrated in greater detail in FIG. 
2E. Specifically, the rollout and support of SEPG projects in step 240 comprises the 

1 5 steps of identifying new projects, step 241 ; assigning a SEPG haison, step 242; 

conducting a project kickoff, step 243; approving or disapproving waivers, step 244; 
coUectmg project metrics, step 245; conducting best practices reviews, step 246; 
reporting best practices status, step 247; and conducting project close out, step 248. 

Referring to FIG. 2E, during the identification of new projects in step 241, the 

20 organization should identify new projects that are in the planning stages. Then, in 

step 242, a SEPG rollout team leader assigns a SEPG liaison by evaluating the current 
workload among the available SEPG liaisons and select the most appropriate SEPG 
haison for the current project. The rollout team leader preferably discusses the 
assignment with the SEPG Haison and sends a memo to tiie SEPG haison informing 

25 him or her of the assignment, with a copy to the project manager and the SEPG 

program leader. 

Continuing with FIG. 2E, the next step of conducting a project kickoff in 
step 243 is conducting a kickoff meeting, preferably within 2 weeks of notification of 
support to be provided by SEPG. The SEPG liaison should schedule a time to meet 
30 with the project management team to discuss the kickoff The SEPG liaison should 

also ask a project manager for project docimientation such as the proposal, the 
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statement of work, estimating tool estimates, and the workplan, if available. This 
discussion is to establish the project organization, identify projects to support, and to 
ascertain the scope of the SEPG effort. 

Referring again to FIG. 2E, the next task in the rollout and support of SEPG 
5 projects in step 240 is to approve or disapprove waivers, step 244. In particular, the 

SEPG liaison or SEPG manager should work with the project or proactively identify 
any area where a project may need a waiver. A waiver request template should be 
available through the SEPG or through the SEPG Haison. A senior management 
official should sign the waiver request form, thereby acknowledging its risk and 

1 0 impact to the project. Also, the SEPG liaison should review the waiver request form 
for completeness and determine the disposition of the waiver request. The SEPG 
haison then forwards the waiver request form to the SEPG project manager with a 
recommendation for disposition. Subsequently, the SEPG liaison informs the project 
manager of the disposition of the waiver request. 

1 5 Continuing with FIG. 2E, the next task is to collect project metrics, step 245. 

Step 245 may include one or more undertaking that help to ensure that the project 
metrics are collected in an organized and efficient manner. These undertakings may 
include collecting monthly project metrics, collecting best practice reviews, collecting 
quality review results, collecting stakeholder scorecard data, and collecting people 

20 satisfaction survey results. 

Continuing with FIG. 2E, another step is to conduct best practices reviews, 
step 246. With the conducting of best practice reviews, the SEPG liaisons should 
conduct monthly best practice reviews with project management in order to track and 
monitor compliance with CMMI requirements. The review criteria are based on the 

25 CMMI process areas and can be found within a best practices matrix. The reviews 

identify nonconformance items and areas for improvement. The SEPG Uaisons 
should review the information gathered fi-om the team and enter comments into the 
notes/comments section of the first best practice review matrix. During the meeting, 
the SEPG liaisons and project managers should review the matrix and determine 

30 which items have been met and those that would require additional information or 

documentation (artifacts). Based on the review, the SEPG liaisons should complete 
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the best practice matrix with documentation on additional information required from 
the project. Once the project reaches substantially complete compliance with the 
identified best practices, the best practice review focus becomes one of continued 
comphance and includes project team leaders and project team members. The SEPG 
5 liaisons should document and spot-check areas for compliance based on past reviews. 

These interviews may be conducted with or without project management in step 500, 
described below. 

Continuing with FIG. 2E, the next step is to report best practice status, 
step 247. There are two ways to report on the best practice status. The SEPG liaison 

10 may document Non-Conformance Items (NCI) and issues in a best practice 

notes/comments section and submit this to project management after the conclusion of 
the Best Practice review, preferably within two days. The project manager generally 
has a short time, such as one week, to provide a response for each NCI, including a 
target completion date for correcting the NCI. Alternatively, the SEPG haison may 

1 5 complete a best practice "Dashboard" report with updated scores, open items, and 

risks. The report should be sent to project management and SEPG rollout and 
program leaders. 

Continuing with FIG. 2E, the next step is to conduct project close out, 
step 248. The organization may implement phase close out. In phase close out, the 

20 SEPG haison may approve or disapprove the waivers, collect project metrics, conduct 
best practice reviews, and report on best practice status, etc. This process of rolling 
out and support of SEPG projects, along with the control of process improvements 
(step 203) described below, may then be repeated until the project is closed out. Next, 
in project closed out, the SEPG liaison works with the project manager and the 

25 management team to evaluate the overall impact and value of the SEPG program on 

the project. This evaluation should be done through the completion of a project close- 
out memo, verification of updates to the internal corporate resource by the project 
knowledge champion, verification of submission of the project's actual and estimated 
values to owners of the estunating tool via the profiling tool, collection of final 

30 project metrics, and collection of best practice and SEPG suggestions and comments. 
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Following the conducting of the close out in step 248, the organization may 
complete the SEPG projects in step 290, as depicted in FIG. 2J. To complete the 
SEPG projects in step 290, the SEPG Liaison reviews the Closing Memo and SQA 
Debrief created by projects that are complete and no longer require SEPG support. 
5 Specifically, the organization verify the completion of the supported projects, review 

the documentation produced in steps 230 and 240, and generate a Ust of best practices 
as desirable to produce a more mature product, respectively steps 292-296. 

Referring back to FIG. 2A, another task of the management and improvement 
process, step 202, in the project stage 200 is to control process improvements, 
10 step 203. The control of process improvements in step 203 brings together the tasks 

associated with controlling and conducting process improvement. As highlighted in 
the methodology, the Control Process Improvement, Rollout & Support SEPG 
% Projects, and Complete SEPG Projects tasks are iterative in nature. Once processes 

~ and tools are improved, the SEPG is responsible for delivering these new processes 

=j 15 and tools to its projects. Improving the control process in step 203 is comprised of the 

following steps: conducting a super software quality assurance (SQA) review 
(step 250), conducting mini-assessments and appraisals (step 260), conducting 
intermittent surveys (step 270) and conducting process improvements (step 280). 

One aspect of improving the control process in step 203 is to conduct a super 
20 SQA review, step 250. In step 250, the SEPG plans and organizes a Super Software 
Quality Assurance (SQA) review of its docimients. A report is prepared based on the 
findings and reviewed with the SEPG Team. The results of this review help the 
SEPG to improve internal processes. The organization performs this step 250 to 
conduct software process and work product quality assurance reviews to verify 
25 project adherence to standards and procedures, such as any identified best practices. 
The quality program section of the Project Plan is described above in greater detail 
within the text accompanying FIG. 2B. Turning to FIG. 2F, the process of conducting 
a super SQA review in step 250 is described in greater detail. The first task of 
conducting the super SQA review in step 250 is to complete a SEPG project plan. In 
30 step 25 1, the process improvement (PI) team leader typically (1) identifies documents 

and processes to be reviewed; (2) ensures that documents in the Project Plan and 
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Work Plan are consistent; (3) identifies Super SQA reviewers, reviewees, and review 
criteria, (4) identifies roles and responsibilities; (5) identifies SQA metrics; (6) 
references the SQA plan in the quality section of the project plan; and (7) creates the 
SQA Plan. 

5 In the next step of conducting the super SQA review of step 250, the 

organization prepares for the super SQA review, step 252, as depicted in FIG. 2F. In 
one implementation of step 252, the PI team leader sets the super SQA review 
expectations. The PI team representative also submits reminder notifications to the 
super SQA reviewer based on any scheduled super SQA reviews, provides the Super 

1 0 SQA Reviewer with the Super SQA Reviewer training presentation and the SEPG 

Program SQA Plan, and provides the super SQA reviewer with standards and 
supporting documents to be reviewed. The PI team representative may further 
provide the super SQA reviewer with document owner contact and availability 
information, as defined in the CMML The super SQA reviewer then typically gathers 

1 5 and reviews criteria/standards and supporting documents fi:om the PI team 

representative, reviews any super SQA reviewer training presentation, and schedules 
meetings with document owners. 

Referring to FIG. 2F, the next step of conducting the super SQA review of 
step 250 is to conduct the super SQA review, step 253. The super SQA reviewer 

20 should review processes and documents against review criteria/standards, conduct 

interviews with document owners, identify nonconformance items, and follow up with 
the document owners as needed for meeting with the requirements of the desired 
CMM level. The document owner participates in the interview with the super SQA 
reviewer and remains available to answer questions. The PI Team Leader should also 

25 remain available to answer questions. 

In the next step 254, the organization, through the SQA reviewer, prepares an 
SQA report to document a detailed summary of findings and recommendations, as 
illustrated in FIG. 2F. Specifically, the SQA Report should include an item number, 
the date reported, and an accurate description of nonconformance items. The SQA 

30 reviewer may further distribute the SQA Report to the PI Team Leader, and schedule 

discussion of nonconformance items with the SEPG Program Lead. The PI team 
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representative also prepares and documents responses in the SQA Report, including 
an indication of whether the PI team representative agrees or disagrees with the 
reason statement, or otherwise determines the jSndings to be not applicable to the 
particular organization or project. 
5 Subsequently, the organization should discuss nonconformance items, 

step 255 in FIG. 2F. In step 255, the super SQA reviewer typically schedules and 
conducts a discussion of nonconformance items with the PI team leader, as well as 
verifying an adequate resolution of nonconformance items. Likewise, the PI team 
leader should discuss nonconformance items with the Super SQA Reviewer and refer 

10 disagreement items for facilitation to the SEPG Program Leader. A PI team 

representative should update the SQA report with proposed resolution(s) and 
projected completion date(s) for proposed changes/actions, and update and return the 
report, as well as all necessary documents, to the SQA reviewer for verification. The 
PI team representative further creates the System Investigation Requests (SIRs) and/or 

1 5 Change Requests (CRs) as necessary for the CMMI. The SIRs correspond to reports 

created to document errors in the product, and CRs conversely correspond to 
enhancements in the product that are beyond the scope of the original product. A 
SEPG liaison may then review and resolve escalated nonconformance items. 

Returning to FIG. 2F, the next step of conducting the super SQA review of 

20 step 250 is to track the super SQA metrics, step 256, by having the super SQA 

reviewer send the final report to the PI team leader. The PI team leader then forwards 
the final report and metrics to SEPG program leader, including metrics such as SQA 
schedule variance, and the number of nonconformance items. The PI team leader 
further tracks and reports on all open nonconformance items, while keeping project 

25 copies of documentation/reports. 

Returning to FIG. 2A, the next step in improving the control process in the 
step 203 is to conduct assessments, step 260. In step 260, the SEPG coordinates 
activities to determine the state of an organizations processes and practices. This 
assessment can take many forms and can range from informal process assessments, 

30 mini-appraisals or full-scale evaluations. In any of these situations the organization 

can utilize outside contractors to conduct the review. Step 260 generally includes 
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three stages: preparation, an on-site period, and wrap-up. After a series of interviews 
and review of documentation, the assessment results are then presented back to the 
organization. The organization should follow the same basic process when 
conducting an appraisal. In both cases, the organization may utilize an external group 
5 to execute the mini-appraisal and/or assessment. The process of conducting the mini- 

assessments and appraisals in step 260 is more fully illustrated in FIG. 2G. 

As depicted in FIG. 2G, the first task in the assessments in step 260 is to select 
projects, step 261. In step 261, the organization c^efiilly selects projects used for the 
assessment in order to paint an accurate picture of the organization's processes. 

10 Generally, from one to four projects should be selected for assessment. Projects may 

be selected using the following criteria: (1) the project should be representative of the 
work (present and fixture) of the organization, and aligned with the business objectives 
of the organization; (2) the project should have at least six people working on it; 
(3) the project should have a duration of greater than three months; and (4) the project 

1 5 should not have a critical activity or milestone during the on-site period. 

Additionally, at least one of the projects should be in the build stage. Personnel from 
the selected projects should also be available for interviews and presentations. 

Returning to FIG. 2G, the next task in the mini-assessment and appraisal is to 
assess the development of an onsite schedule, step 262. The core of the assessment 

20 during step 260 is made up of the onsite period, which usually lasts JBrom five to ten 
days. The onsite period consists of three basic activities: (1) gathering information 
through interview sessions with project leaders, team leaders, and functional area 
representatives; (2) mapping information to processes areas within the scope of the 
assessment through consolidation sessions; and (3) reporting findings and 

25 observations back to the organization through preliminary and final findings 

presentations. An executive session and a debriefing session is conducted to wrap up 
the on-site period. There is no limit to the number of hours spent on a particular 
activity; however, the assessment team is bound to the tasks that need to be completed 
before the next day. Training the assessment team is the other activity that can be 

30 considered part of the onsite period, as requned, and can be scheduled just before the 

assessment activities begin. 
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As depicted in FIG. 2G, the next step in step 260 is to prepare assessment 
logistics, step 263. During step 263, a local assessment coordinator works with the 
assessment tecun leader to identify and prepare the logistics for conducting the on-site 
period. Logistical preparations include reservation of rooms for the on-site period 
5 (presentation, interview, and assessment team rooms); computer and presentation 

equipment (projectors, LAN connections, access to a phone, printer, copier, and 
general supplies); arranging for food and beverages, as well as accommodations for 
the assessment team, and confirming building/office access for the assessment team 
during the on-site period. 

10 The next step in step 260 is to select assessment participants, step 264, as 

depicted in FIG. 2G. In the selection of assessment participants in step 264, a good 
^ cross section of the organization must be considered when selecting assessment 

participants. This is done through interviewing individuals firom each selected 
project, including the project leader, project team members, as well as personnel from 

15 supporting groups, such as quality assurance, configuration management, and/or the 



20 



database group. Individuals who have been involved in developing or maintaining 
software in the organization also should be included in the hst of interviewees. 
Participants selected for the assessment should come from different parts of the 
project life cycle, have at least six months of experience with the organization (and at 
least three months with the project), and be able to articulate their observations and 
opinions about the organization and its projects. Selected participants preferably can 
dedicate from six to eight hours to the assessment activities during the on-site period. 
For the assessment to take place, the assessment sponsor must be present for the 
initial and final presentations. 



25 



Returning to FIG. 2G, the next step in the mini-assessment and appraisal is to 
develop organizational awareness of CMMI, step 265. The organization perfortris this 
task to train assessment participants on what the assessment will involve, how the 



assessment will be conducted, and what the organization expects of assessment 



participants. Awareness activities may include training sessions and distribution of 



30 



awareness materials to everybody in the organization. The assessment sponsor, or the 
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organization's management (if different from the sponsor), must demonstrate their 
total support for the initiative. 

As depicted in FIG. 2G, the next step in the mini-assessment and appraisal is 
to collect process information, step 266. Step 266 is performed in preparation for the 
5 assessment of the collection of the documentation used in the current management 

and technical processes. Selected members of the organization fill out a maturity 
questionnaire to provide a baseline for scoping the assessment. The appropriate 
process documentation from both the organization and the projects being assessed 
should be collected to be reviewed by the team for the purpose of developing the 

10 assessment findings and observations. A documentation index should be created and, 

if required, the collected documents should be mapped to the CMMI process areas. 

Returning to FIG. 2G, the next step in step 260 is to conduct the assessment, 
step 267. In step 267, the assessment team visits the organization with the objective 
of mapping the organization's management and development processes against the 

15 CMMI. In particular, the assessment team should be trained. The assessment team 

should further conduct an opening meeting and interview project leaders, team 
leaders, and functional area representatives. The assessment team should further 
review collected documentation and consolidate information gathered and map it to 
process areas in the CMMI. Subsequently, the assessment team should conduct 

20 follow-up interviews, as required, and prepare and present preliminary findings to 

management and staff. Likewise, the assessment team should prepare and present 
final findings to the organization, incorporating feedback received from the 
preliminary findings presentation. The assessment team then conducts any executive 
and debrief sessions and prepares a final report. At the conclusion of the assessment, 

25 the assessment team files an assessment report with the Software Engineering 

Institute (SEI), including with the assessment the final presentation and the summary 
report. 

Returning to FIG. 2A, the next step in improving the control process in 
step 203 is to conduct a quarterly survey, step 270. The organization performs step 
30 270 to receive feedback from projects regarding the SEPG processes and tools. 

During step 270, the SEPG designs and delivers a quarterly Process Improvement 
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Svirvey to the projects it supports. The results of this survey are an input into the 
SEPG team's Process Improvement efforts. While named a quarterly survey, it 
should be appreciated that the survey may occur at other intervals and time periods. 
Results of this survey should be used to improve SEPG processes and tools. The 
5 process of conducting the quarterly survey in step 270 is depicted in FIG. 2H. The 

first task is to develop a survey process, in step 271, for administering the process 
improvement survey. This survey may be administered by the SEPG. The 
responsibilities for this task should be assigned to a sub-team. In developing the 
survey in step 271, the organization should consider the effect of the survey 

10 transmission medium and the methods through which the survey results will be 

analyzed. The organization should next develop the survey questions, step 272. The 
organization should select question on which the SEPG would like to receive 
feedback. Preferably, when developing survey questions, the organization should 
choose nonleading questions. The organization should also preferably use a response 

15 scale that can be easily quantified, such as the Lickert scale. The organization should 

next, in step 273, administer the survey using the medium chosen in step 271 . At this 
point, in step 274, the organization may evaluate and analyze the survey results 
received from respondents using the process developed in step 271. The organization 
may then use the survey results to improve the SEPG processes and tools, step 275. 

20 During step 275, the organization may also publicize the results of the survey. 

Returning to FIG. 2A, the next step in improving the control process in 
step 203 is to conduct process improvements, step 280. The organization performs 
step 280 to manage process improvement activities. During step 280, the SEPG takes 
the feedback it received from the SQA review, assessments, quarterly surveys, and 

25 feedback from other sources, and begins translating this feedback into improvements 

in SEPG processes and tools. Results from step 280 will be used to maintain, update, 
and develop SEPG processes, tools, and assets. Step 280 is illustrated in greater detail 
in FIG. 21. 

As depicted in FIG. 21, the first step in conducting process improvements is to 
30 maintain and improve processes and tools, step 281. In step 281, anyone in the 

organization and its external reviewers may identify a process improvement 
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opportunity (with a process, template, training, standard, tools, or the document 
repository itself). This could be in the form of an error (through a SIR), an 
improvement, an enhancement request (through a CR), or any other process 
improvement concern. The process improvement team leader should examine the 
5 process improvement opportunity, and a decision may be made on implementing the 

process improvement. If it is determined that the changes will be incorporated into 
the appropriate process asset (process, template, standard, training, etc.) in accordance 
with SEPG standards, then a SIR or CR may be documented to capture the change. 
Returning to FIG. 21, tiie next step in conducting process improvements, 

10 step 280, is to define and update processes and tools, step 282. In step 282, anyone in 

the organization may identify a new process or tool to be defined, documented, and/or 
buih. The first type of process to be created is an internal SEPG process that uses a 
new process template to document process flows and descriptions. The second type 
of process to be created includes processes and tools that are part of the SEPG 

15 methodology. Process definition is submitted to a SEPG team leader for review and 

approval. If a process is approved, it will be scheduled for release to the organization 
and/or SEPG team, dependiag on the type of process submitted. 

As depicted in FIG. 21, the next step in conducting process improvements, 
step 280, is to pilot processes and tools, step 283. Once the process or tool to be 

20 piloted has been completed and approved, it is time to determine the pilot group, time 

firame, scope and fimctionahty, roles and responsibihties, and entry and exit criteria, 
as part of step 283. The SEPG program manager may then work with a process asset 
owner to communicate the pilot's scope and expectations with the pilot group. The 
pilot group will be trained on the use and implementation of the process asset. The 

25 pilot is conducted with the process asset owner providing support to the pilot group in 

terms of providing clarification, additional training, or technical support, as necessary. 
At the end of the pilot period, the process asset owner debriefs with the pilot group or 
at least with a representative of the group to evaluate the pilot. Strengths and 
weaknesses of the process are identified, documented, and addressed. 

30 Retiuning to FIG. 21, the next step in conducting process improvements, 

step 280, is to roll out processes and tools, step 284. In step 284, once feedback from 
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the pilot group is incorporated into the process asset, it will be rolled out to the 
organization and/or SEPG team, as necessary. In step 284, the SEPG Uaisons have 
the primary responsibihty of communicating the new processes and tools to the 
organization's projects. 

Returning again to FIG. 21, the last step in conducting process improvements, 
step 280, is to assess and evaluate processes and tools, step 285. During step 285, the 
organization determines how processes and tools will be evaluated. The organization 
may ftirther conduct intermittent or quarterly process improvement surveys. 

Personnel Stage 

Returning to FIG. 1, a second process within the organization management 
step 100 relates to personnel management, step 300. The actions of step personnel 
management in step 300 generally relate to acquiring, organizing, and training the 
organization's personnel as needed to encourage the development of more mature 
products and achieve higher levels of CMM maturity. Organizational Training of step 
300 is generally necessary to enable personnel to develop skills to meet specific roles 
and responsibilities during solutions delivery in step 600 described below. The 
process of personnel management is generally depicted in FIG. 3 A and comprises the 
actions of designing a performance measurement infirastructure, step 310; executing 
organization design and development, step 320; and designing and deploying training, 
step 330; and is now briefly described. 

The designing of a performance measurement infrastructure in step 310 
generally relates to planing activities related to performance measurement to provide 
the organization with a means forjudging the effectiveness of the organization. The 
designing a performance measurement infirastructure in step 310 is summarized in 
FIG. 3B. The first step in step 3 10 is to vaUdate and reach agreement on organization 
strategy, step 312. Step 312 generally involves the organization's key stakeholders in 
the development and/or validation of the organization's strategy, specifically the 
organization's mission, vision, and overall objectives. Because performance 
measurement cascades down from the organization strategy, the organization's 
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strategy should be understood and agreed upon by those accountable for 
implementing it. 

Returning to FIG. 3B, the next step in designing a performance measiirement 
infrastructure in step 310 is to produce a performance measurement scorecard, 
5 step 314. In step 314, the organization uses a balanced scorecard to measure 

performance. The balanced scorecard is a measurement tool that translates strategic 
objectives into a coherent set of performance measures. The scorecard is "balanced" 
because it measures both leading and lagging indicators. These indicators are 
expressed in financial and nonfinancial terms. 

10 Subsequently, in step 316, the organization implements the scorecard, as 

depicted in FIG. 3B. Implementing the scorecard generally requires that the specific 
information for performance goals, metrics, and targets be collected from the front 
lines. Furthermore, the organization should compile at the strategic level each 
performance perspective, objective, metric, and target. Also, the organization should 

1 5 create and communicate top down, bottom up, and interactive performance measures. 

Subsequently, the organization should solicit feedback to test the effectiveness of 
metrics and how the performance measures fit in with the organization strategy. 

Turning to FIG. 3C, the next step in the personnel stage 300 is to execute the 
organization design and development, step 320. The organization performs step 320 

20 to plan activities related to organization design and development. Step 320 involves 

coordinating the tasks associated with defining a strategy for the organization, 
assessing the organization against this strategy, and deigning and implementing a new 
organization. Note that step 320 assumes that the organization has a Human Resource 
Organization with the skills to design and implement new elements of the 

25 organization. The organization should to have experience in organization design and 

development. The substeps of the organization of design and development in step 320 
are illustirated in FIG. 3C. 

The first task in step 320, as illustrated in FIG. 3C, is to identify an 
organization strategy, step 321. In step 321, business outcomes, core competencies 

30 and guiding principles are defined. These definitions will position the organization 

relative to business goals and objectives, vision and mission, management philosophy. 
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customer values, critical behaviors and competitive environment. Specifically, the 
organization should identify an organization strategy before detailed org^zation 
design. The organization should be designed to reflect not only where the company is 
relative to strategy, philosophy, and the value proposition of its customers, but also 
5 where it needs to achieve a competitive advantage in the future. The organization 
strategy sets the direction by defining business outcomes, core competencies, and 
guiding principles that will be used to anchor the organization design and 
development process. 

As depicted in FIG. 3C, the next step in the execution of the organization 

10 design and development in step 320 is to conduct an organization assessment, 

step 322. It should be noted that the assessment differs from assessment used with 
SEPG. The organization assessment helps to identify the supports and barriers to 
transformation and build a case for implementation. An organization assessment in 
step 322 consists of assessing an organization's current situation, its future 

15 aspirations, and the gap between them; then identifying the initiatives required to fill 

these gaps. In step 322, enablers and barriers to organizational transformation ^e 
identified and a case for implementation is built. This is accomplished through an 
assessment of the current organizational environment and future organizational 
aspirations, identifying the gaps between these two, and identifying a course of action 

20 to close those gaps. 

Referring to FIG. 3C, the next task in step 320 is to design an organization 
infi-astructure, step 323, to create structures estabhshed to form individuals into the 
desired performing organization. The organization infrastructure's goal is to allow 
workers to effectively accomplish their tasks within the business process so that an 

25 overall goal is met. In step 323, the organization will design a competency model and 

design roles, jobs, teams and organizational structures. The competency model 
definition will document the knowledge, skills and other attributes/abiUties associated 
with high performance on a job. The roles, jobs, teams and organizational structures 
will docimient the responsibilities associated with: the individual (roles), groups of 

30 related roles Gobs), groups of jobs (teams) and the span of control, reporting 

relationships and functional relationships of all of these components. Step 323 has 
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two subtasks — to design a competency model, step 324, and to design roles, jobs, 
teams and an organization structure, step 325. Steps 324-325 may be conducted 
iteratively and/or concurrently. In designing a competency model in step 324, the 
organization should group together related competencies to form a competency 
5 model. A competency is a cluster of related knowledge, skills, and other 

attributes/abilities associated with high performance on a job; and a competency 
model is a group of related competencies required to perform a career field such as 
team leader or technical coach. Similarly, during the process of designing an 
organization structure in step 325, the organization defines the roles played by 
1 0 individuals, the jobs they hold, the teams in which they work, and the relationship 

i=a between teams. The organization should logically define roles for individuals on the 

S basis of their competencies, as decide in step 324. 

'ri Returning to FIG. 3C, the next task in step 320 is to verify and validate an 

'%! organization structure, step 326. In step 326, all components of the newly defined 

^ H 1 5 organizational uifrastnicture and reviewed to verify and validate that they meet the 

needs and goals of the organization. Specifically, the organization should verify and 
fu validate that any new organization design meets the needs of the business and is 

"j internally consistent. The organization should further confirm the new organization 

^ J design with any subject matter experts and initiative sponsors. Continuing with 

20 step 326, the organization should organize review sessions to validate how well the 

components of the new organization design (roles, jobs, teams, organization 

infi-astructure, performance management infi-astructure) fit together to support new 

initiatives. 

The next task in step 320, as illustrated in FIG. 3C, is to design a performance 
25 management infirastructure, step 327. In step 327, the organization's performance 

measurement scorecard is developed based on the organization's strategic objectives. 
This scorecard is then used to measure the organization's performance. Note that this 
task assumes that the organization has a Human Resource Organization with the skills 
to design and implement a performance measurement scorecard, and that the 
30 organization has experience in organizational performance management. Thus, in 

step 327, the organization defines a means for assessing, rewarding, and developing 
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the individuals in an organization. The performance management infrastructure has 
four components: (1) designing the performance management approach; (2) designing 
the performance appraisal instruments; (3) designing career progression; and 
(4) designing the compensation and reward structure. Overall, the organization 
5 should establish a system to reward individuals for desired contributions. 

The final task in step 320 is to determine an organization infrastructure 
mobilization approach, step 328, as illustrated in FIG. 3C. In step 328, the 
organization determines and mobilizes the resources required to staff the new 
organization infrastructure estabUshed in step 323. The organization may determine 

10 profiles for the ideal candidates, determine sizing and timing needs, and determine a 

sourcing approach. For instance, candidates may be profiled to fit job descriptions, 
the organizations new size may be determined and an approach to sourcing and 
staffing jobs may be fmalized and executed.. 

Returning to FIG. 3A, the next process in the personnel stage 300 is to design 

1 5 and deploy training, step 330. fri step 33, the training needs of the organization are 

analyzed and a Training Plan is created, training is designed, developed and deliverer 
and post implementation support is provided. The organization performs step 330 to 
plan activities related to training employees. The design and deployment of training 
during step 330 is illustrated in greater detail in FIG. 3D. As illustrated in FIG. 3D, 

20 the first task in step 330 is to conduct a training needs analysis, step 33 1, during 

which the organization identifies, by name, the participants to be trained, along with 
the courses and modules on which these participants will be trained. In step 331, 
target audiences and participants are identified, and training courses and modules are 
planned. The training needs analysis in step 33 1 may be conducted in two phases. 

25 During the first phase, the organization gathers the high-level training needs for the 

organization. Similarly, the second phase consists of gathering the detailed training 
needs for the organization. 

Returning to FIG. 3D, the next task in FIG. 3D is to develop a fraining plan, 
step 332, as needed, to describe the organization's overall fraining approach. In step 

30 332, the overall organizational approach to fraining is docimiented. The fraining plan 

formed in step 332 may include any of following sections/topics: Objectives; 
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assumptions; overall training approach; training courses, modules, and topics; training 
timeline; training logistics; and training evaluation. 

The next task in step 330, as illustrated in FIG. 3D, is to design training, 
step 333. In step 333, the training standards, templates, instructor and participant 
5 guides and the actual layout/format of training are developed. Specifically, the 

organization may develop the layout/format for the training materials. The 
development includes developing training development standards as well as templates 
for any instructor and participant guides. 

Similarly, the next task in step 330, as illustrated in FIG. 3D, is to develop 

10 training, step 334. In step 334, course content is created using the materials compiled 

during the training design step 333. The organization may implement step 334 by 
creating the course content using the training development standards and instructor 
and participant guides. Other material created in step 334 may include "Train-the- 
Trainer" materials, visvials, job aids/handouts, and tracking documents. Using the 

15 training materials developed during step 334, the organization may deUver training, 

step 335, such as a Train-the-Trainer session, a pilot training session, and the actual 
training. 

Returning to FIG. 3D, the next task in step 330 is to provide post- 
implementation support, step 336. In particular, during step 336, the organization 
20 should provide a short-term dedicated staff (e.g., one or two weeks) to support the 
users in applying what they've learned on the job. Furthermore, the support staff 
should be available to answer questions, identify and troubleshoot issues, and share 
best practices. 

Throughout steps 200 and 300, as well as other steps in the CMM in a Box 
25 Method 10, the organization may need to commit to one or more actions (not 

illustrated) as required to achieve higher matiuity levels in the CMM or the CMMI. 
Commit points are major decisions regarding reporting the progress of present work 
and obtaining authorization to continue. Commit points define the boimdaries of each 
stage around key decisions related to content, context and course of action. For 
30 instance, a commit point may be implemented prior to the executing and design of an 
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organization infrastructure in step 320, to require that the design of the new 
organization structure must be approved before further implementation can proceed. 

Program Management 
5 Returning to FIG. 1 , a second primary component of the CMM in a BOX 

method 10 of the present invention is program management step 400. Program 
management step 400 generally concerns activities directly related to the creation and 
refinement of a progrmn for implementing the CMM in a BOX method 10. 
Specifically, program management 400 focuses on the continuous oversight needed to 

1 0 support the delivery of a business solution through multiple projects and releases. 

Appropriate disciplines, techniques, and tools are used in step 400 to plan and 
organize the work, and to manage the incremental delivery of the new business 
solution. As illustrated in FIG. 4A, the program management stage 400 generally 
comprises the steps of justifying the program (step 410); planning the program 

15 execution (step 420); organizing program resources (step 430); controlling program 

work (step 440); and completing the program (step 450). These individual steps are 
now described in greater detail. 

As depicted in FIG. 4A, the organization may first justify the program, 
step 410. In step 410, a Program Business Case is prepared. The program business 

20 case approach is referenced to develop the business case. The business case is 

designed to secure stakeholder support for the program. Topics of the business case 
include the program's understanding of the current problem, the proposed solution to 
the problem that is to be implemented by the program, and a cost/benefit analysis. 
Justification of the program to all key stakeholders and sponsors helps in the 

25 successful execution, implementation and completion of the program. The program 

business case should provide economic justification for the change journey and for 
each program within the change journey. The program business case generally 
explains why the sponsoring organization should change, what value it receives by 
changing, and what steps are necessary for a successful change. The program business 

30 case addresses three main components, including business context and change 
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imperatives, value impact analysis, and change journey. The tasks in the justification 
of the program in step are generally illustrated in FIG. 4B. 

Referring to FIG. 4B, the organization first determines an economic evaluation 
approach, step 41 1, to obtain a "buy in" fi-om the appropriate stakeholders in the 
5 sponsoring organization on the overall implementation approach for the program. 

Specifically, the organization tries to demonstrate the tangible benefits of a program 
to the affected parties. Step 411 attempts to show the process of implementing the 
program as an investment with positive, long-term benefits. 

Returning to FIG. 4B, the next task in step 410 is to create a model structure, 

10 step 412. In step 412, the organization obtains internal agreement regarding the 

structure of the model used to determine the benefits of implementing the program. 
For example, benefits to be derived may be expressed in terms of increased market 
share or reduced operating costs. In this way, affected parties may communicate the 
program's effects in terms of similar measures of costs and benefits. As suggested in 

1 5 FIG. 4B, the organization may also attempt to justify the program by forecasting 

baseline business performance, step 413. In other words, the organization may 
attempt to determine how the organization and its comprising units would perform 
without implementing the program. Continuing with FIG. 4B, another task in 
step 410 is to project net change journey benefits, step 414. The organization 

20 performs step 414 to predict and quantify the benefits that will be derived from 

implementing the program. 

The next step in step 410 is to assemble a business case, step 415, using the 
results assembled during steps 41 1-14. The organization may perform step 415 to 
document rationale for implementing the program. Ultimately, this documentation 

25 may serve as a motivational tool for change within the organization. 

Returning to FIG. 4A, the next task in the program management stage 400 is 
to plan the program execution, step 420. During step 420, the organization develops 
plans for the program itself, financial management and resource management. 
Program approaches axe referenced during the creation of these documents. These 

30 plans guide the continued implementation of the program and are what the program 

will monitor itself against during later tasks. The individual tasks of step 420 are 
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illustrated in FIG. 4C. In step 420, the organization may develop a consolidated 
program plan, which documents the necessary tasks, effort, schedule, and costs for all 
releases of a business capability. The organization may also refine a program 
statement of work, and develop bottom-up project plans. Subsequently, the 
5 organization reconciles these plans with the top-down plans to generate a program 
baseline. The organization may have performed step 420 initially dviring program 
planning, in conjunction with or prior to the analysis stage 700 described below. 
Then, step 420 may be reinitiated during the coiirse of the program as replanning is 
required by program management. 

10 Looking at FIG. 4C, the first task of the plan program execution in step 420 is 

to plan program processes, step 422. The organization may specifically determine all 
the management processes necessary to support the program. These relate to 
resources, vendors, quality, configuration, releases, issues, problems, risk, finances, 
contingency, and performance reporting. The organization may estabUsh and 

15 document goals and metrics for each management process. The organization should 

begin this task package at the start of the program, and refine the management 
processes as the program progresses. The organization may perform this initial 
planning at the program level to help ensure that there are no gaps or overlaps of 
activities. While all the activities within the Delivering phase may be required for a 

20 particular business capability, it is unlikely that all of the activities should fall within 

the scope of a single project team. If the initial distribution of the activities to project 
type is done at the program level, the risk of missing or dupUcate activities is limited. 

Returning to FIG. 4C, the next step in the plan program execution, step 420, is 
to develop a program budget, step 424. In step 424, the organization may establish a 

25 program budget that augments the cost baseline established in the program plan. The 

program budget provides the additional information needed by program management 
to manage the day-to-day financial affairs of the program. 

Another step in the plan program execution, step 420, is to develop a program 
communications plan, step 426, as illustrated in FIG. 4C. In step 426, the 

30 orgMdzation may identify and plan messages to program persoimel, key program 
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executives, and other stakeholders in the program. In that way, step 426 addresses the 
communication needs within the program teams. 

Subsequently, the organization performs the task of finahzing the program 
plan, step 428, as depicted in FIG. 4C. In step 428, the organization may assemble 
5 the composite program plan. The Program Plan compiles the outputs from the plan 
management process 422 with the development of a program plan in step 424. The 
organization may then obtain executive and other appropriate management 
understanding and approval of the fully elaborated program plan and its components. 
The organization ftirther briefs all key stakeholders (i.e., executive management, and 

10 impacted business operations) to ensure their understanding of, and commitment to, 

the program plan. This is crucial, because following this task: (1) the program should 
be described to the organization, and (2) more personnel should be assigned to the 
program. The organization may then take this opportunity to resolve any unclear or 
incorrect stakeholder expectations. 

1 5 Returning to FIG. 4A, the next step of the program management 400 is to 

organize program resources, step 430. During the step 430, resource requirements are 
analyzed and aUgned so as to meet program objectives. As the program determines 
its resource needs, the Program Resource Request is completed to obtain the 
resources. Organize Program Resources is linked closely to planning the program's 

20 execution and pertains to staffing of the overall program. Under the Plan Program 

Execution task, the Program will plan for and deal with resource questions related to 
subordinate projects. Specifically, the organization may generally analyze resource 
requirements, initiate the procurement of goods and services, obtain human and 
physical resources from participating entities, assign these resources to projects, and 

25 release the resources upon project completion. The organization may perform 

step 430 throughout the Ufe of the program created and implemented in step 400. 

As illustrated in FIG. 4D, in order to obtain and deploy resources, the 
organization may analyze resource requirements, step 432. This task analyzes 
resource requirements as defined in a program management resource plan. Resource 

30 requirements are consolidated from project needs and should generally include 

desired resource provider (generally the organization itself), if previously determined. 
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resource skill/type, and time period (such as monthly). Continuing with step 432, the 
organization may create a program resource management plan that forecasts resource 
needs by stage and capability release. 

Returning to FIG. 4D, the organization may ftirther obtain and deploy human 
5 resources, step 434. Human resources are obtained by initiating a request with the 
Human Resources Organization, interviewing potential candidates, and selecting the 
candidate that best fits the requirements. Human resoiirces are then assigned to 
projects as they arrive at the program. This task, alternatively, may be assigned to the 
projects. The program resource management plan may reflect actual information 

10 regarding the resource request. 

Returning to FIG. 4D, the organization may also procure and deploy physical 
resources, step 436. Physical resources are generally procured by initiating a resource 
request, evaluating the potential resources, and selecting the resource that best fits the 
requirements. Resources are then assigned to projects as they arrive at the program. 

15 The Physical Assets Liventory and the Program Resource Management Approach are 

generally both updated to reflect actual information regarding the resource request. 

Referring again to FIG. 4D, the organization also releases resources, step 438. 
When human resources are assigned to projects, they receive a "roll-off date 
indicating when these himian resources are eligible for reassignment within or outside 

20 the program. If not reassigned inside the program, human resources are released to 

appropriate human resources departments for reassignment. Similarly, physical 
resource utilization is scheduled by each project, and these resources are returned 
upon completion of use. This process generally coincides with the completion of each 
stage of work. At that point, a determination should be made whether to retain or 

25 release the human and physical resoxirces from the program. At this point in 

process 430, the entire process 430 may repeat if there are more program resources to 
organize, decision 439. 

FIGS. 4A and 4E illustrate another step in the program management process, 
the control of program, step 440. In step 440, program management monitors 

30 program performance against program plans. Deviations from the plan are monitored. 

Corrective action is taken to resolve deviations as necessary. Program plans are 
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updated to reflect modifications to the program. Step 440 generally provides 
leadership to guide the planning and execution of program work. In step 440, the 
organization may maintain key working relationships within the program, while 
monitoring and developing the skills and performance of program management team 
5 members. The organization may further identify and assess problems with program 

performance, and specify corrective actions as needed. The organization may 
evaluate program metrics to determine progress toward program objectives, and to 
determine whether or not the current metrics are still relevant. The organization may 
further assess whether or not the program is on track by reviewing program, project, 

1 0 and vendor performance. 

The first task of step 440 is to administer the program, step 441 as illustrated 
in FIG. 4E. An effective program administration results in a planned, organized, and 
managed program management office performing a wide range of cost-effective 
activities. As required, the teamwork environment requirements Ust deliverable 

1 5 should be updated to reflect relevant changes in the program. Program leaders should 
also strive to maintain a culture that encourages program participants to achieve 
maximum results. Program leaders should also communicate the common program 
vision to inspire others to support program goals. 

A second task in step 440 is to report performance, step 442, as illustrated in 

20 FIG. 4E. The organization may process and prepare reports for cost/schedule and 
other performance data (e.g., quahty, risk, resoiu-ce, etc.). This should involve a 
standard set of reports as defined in the program performance reporting approach 
section of the program plan. Any ad hoc reports requested by program management 
may also be prepared. 

25 Returning to FIG. 4E, another task in step 440 is to perform financial 

management, step 443. Specifically, the organization may report, monitor, and 
account for the program's fmancial performance and results by performing the 
financial management functions as specified in the Program Financial Management 
Approach section of the Program Plan. Similarly, in step 444, the maintenance of 

30 administrative policies and standards, the organization may update and refine the 

administrative policies and standards on the basis of their effectiveness and the 



-39- 



Atty. Dckt No.: 86769-006 



evolving needs of the program, as illustrated in FIG. 4E. The organization should 
further communicate the changes to the program team members. 

Returning to FIG. 4E, the next task in step 440 is to conduct, as necessary, 
ongoing program orientation and training, step 445. In step 445, the organization may 
5 conduct periodic orientation and training sessions as new members join the program, 

as new types of training are required, and as team members need additional career 
development opportunities. Likewise, in step 446, the organization monitors a 
program communications plan to help to ensure that the appropriate groups 
accomplish their responsibilities. The program management office itself may also be 

1 0 responsible for performing some of the activities as directed in the program 

commimications plan. 

In another group of steps illustrated in FIG. 4F, the organization may complete 
the program, step 450. In step 450, a program closeout report is prepared along with 
other program closeout docimientation. The program is demobilized and 

15 responsibility for the program is transferred to the necessary parties. The organization 
achieves an orderly and successful program closure by formally transferring 
responsibility for the solution components to the operational units, obtaining formal 
management acceptance of the competitive solutions delivered, releasing the 
remaining human and physical resources to their providing organizations/owners, and 

20 completing a disposition of all program documentation and other materials. 

As illustrated in FIG. 4F, one step in the completing the product is to complete 
documentation, step 452. The activities needed to complete all program 
documentation include preparing any final documentation needed to close the 
program, including final cost, performance reports, etc. Additionally, a final review 

25 of the documentation is performed in step 452 to ensxjre that it is complete and 

conforms to program standards. The organization should also identify materials that 
should be shared across the organization, especially process improvements, 
methodologies, techniques, estimating models, and reusable components. The 
organization should also take steps to ensure that the materials are included in the 

30 appropriate repositories. The program documentation and other materials are 

transferred to any appropriate locations. Key deliverables are sent to the software 
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engineering process group team, as determined. A summary of the program's final 
disposition, assets, records, and other appropriate, relevant information should be 
contained in the program closeout report deliverable. 

Continuing with FIG. 4F, another step in the completion of the product is to 
5 transfer program responsibility, step 454. This activity transfers responsibiUty for the 

business capability to the appropriate organizational umt(s). Responsibility is 
assumed by the organizational units responsible for the continuing operation, 
maintenance, and use of the business capabiUty and its underlying components. 

Returning to FIG. 4F, another step in the completion of the product is to 
10 demobilize the program, step 456. The resources to be released include the remaining 

program participants and all facilities (including furniture and equipment). The human 
resources are retumed to the organizational units that provided them. The physical 
resources are released or retumed to their owners. Any remaining procurement 
agreements (purchase orders, contracts, leases, rental agreements, etc.) are closed out. 

15 

Project Management 

Returning to FIG. 1, the CMM in a BOX method 10 generally calls for the 
organizations to concurrently perform project management 500 with the program 
management 400. The project management 500 is generally depicted in FIGS. 5A- 

20 50. Project management 500 generally concerns activities and structures directly 

related to the creation and refinement of a project or product for sale. Project 
management 500 controls the delivery of the specific components from which a 
business solution is derived through the balanced management of Scope, Quality, 
Effort, Risk and Timehne (SQERT). Project management 500 focuses on making 

25 critical decisions and managing risk that will ensure the delivery of the promised 

scope, on time and within budget at the agreed-upon levels of quahty. When a 
program management fimction exists, project management works closely with 
program management to execute the SQERT activities in relation to the delivery of 
multiple projects under one overall program. As illustrated in FIG. 5 A, project 

30 management 500 generally includes planning of project execution (step 510); 

organization of project resources (step 520); control project work (step 530); 
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completion of the project (step 540); an SQA review execution (step 550); and 
supplier agreement management (step 560). 

FIG, 5B presents the individual tasks required in the planning of project 
execution, step 510. The organization may perform this task package 510 at project 
5 initiation to define pieces of the initial Project Plan and subordinate plans that should 

be used to manage the execution of the project. The tasks associated with Plan 
Project Execution, such as planning and estimating, are performed throughout the 
project lifecycle at predefiined decision points, and whenever replanning is required. 
Dviring the planning of project execution in step 510, the organization may tailor the 

1 0 process, step 5 12, to suit a project's needs by i^ing known tools or means. The 

organization may further request a waiver for any required steps that should not be 
followed on the particular project. 

The organization may further implement the planning of project execution, 
step 510 through the development of a project plan, step 514, as illustrated in 

1 5 FIG. 5B. The organization may perform this task using a template to customize a 
specific project. The project plan describes the project approach for the project 
timetable, metrics, organization, supplier agreement management, communication and 
sponsorship strategy, training, quality initiatives, software system development 
process, configuration management, logistics, facilities, tools, and piirchasing. The 

20 project plan also describes the project approach for training, metrics tracking, and 
roles and responsibilities on the project. The organization may fiirther use a best 
practices matrix, a metrics plan, a DAR reference document, and a training needs 
matrix to develop the project plan, as defined in the CMMI. The DAR reference 
document describes the formal DAR process and provides guidelines for identifying 

25 DAR triggers, setting thresholds, and selecting the best techniques. This information 

should be used to complete the quahty program section of the project plan. The 
metrics plan generally contains the hst of required and recommended metrics that a 
project should include in the project plan. 

The planning of project execution, step 510, continues in FIG. 5B with the 

30 development of subordinate plans, step 516. In step 516, the organization may 

develop the appropriate subordinate plans to satisfy the needs of the project. For 
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instance, the organization may define, as needed, subordinate plans for subcontractor 
management, risk management, communication and sponsorship, and configuration 
management. All projects require the creation of a work plan, and an organization 
may create a bottom-up or task-level project work plan based upon estimates. Critical 
5 paths and dependencies are defined and managed within the project work-planning 

tool, such as the Microsoft Project and Project Workbench®. 

Returning to FIG. 5B, the next step in plan project execution 510 is to develop 
project estimates, step 518. The development of project estimates in step 518 is 
analogous to the development of project estimates in step 218, as described above in 

10 FIG. 2B. Specifically, the organization may develop project estimates, step 518, 

using an estimating tool as a starting point for the estimates. For instance, estimates 
may be developed using the following steps: (1) tailor tasks and estimating model; 
(2) determine estimating factor values; (3) define work packages; (4) determine a 
timeline for the estimate; (5) reconcile a present estimate to an initial estimate; and 

1 5 (6) document assumptions used to form the estimates. The organization preferably 
fiirther validates any estimates by verifying estunates against estimates or actual 
results from comparable projects. To form accurate estimates of available resources, 
the organization should further consider other resource-tapping activities, such as 
community involvement, recruiting, mentoring, and training, when evaluating 

20 resources. 

Another step of the project management 500 is to organize project resources, 
step 520, as illustrated in FIG. 5C. The organizing of project resources in step 520, as 
well as in substeps 521-25, are analogous to steps 220-25, described above in FIG. 
2C. The organization can perform these tasks as needed to organize the project's 

25 human resources, establish other resources, to make work assignments, and to develop 

training enabling resources. In step 520, the project focuses on obtaining, assigning 
and training its himian resources and establishing the project's other resources. This 
task is performed iteratively as needed to organize, mobilize and manage project 
resources throughout the execution of the project. 

30 Turning to FIG. 5C, the first step in organizing the project resources in 

step 520 is to refine resoxjrce needs, step 521. In this step 521, the organization 
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defines the team organization structure, schedules the work, and defines the human 
and physical resource needs of the project. These tasks are performed in view of each 
project's requirements. By refining resource needs in step 521, the organization helps 
to ensure that project staffing and facilities needs are met on a timely basis without 
5 affecting the completion date and the quaUty of the work. The organization may 

complete this refining of resource needs in step 521 by (1) determining project 
organization structure; (2) balancing a development schedule using human resource 
guidelines; and (3) refining physical resource needs that were outlined in the logistics, 
facilities, and tools section of the project plan formed in step 210. 

10 Returning to FIG. 5C, the organization continues the organization of the 

process resources in step 520 by establishing project standards and goals, step 522. 
The establishment of project standards and goals in step 522 is accomplished by 
developing, modifying, and adopting administrative and project-specific project 
standards and procedures. Examples of administrative procedures are employee 

1 5 availabiUty checklists, time accounting procedures, status reporting, vacation 

scheduling, etc. Project standards and procedures include design and development 
standards, and the use of project-specific tools. The estabUshment of these standards 
and procedures preferably improves the organization's communication, operating 
efficiency, and overall control of the project. 

20 The organization continues the step of organizing the process resources, 

step 520, through organizing a project team in step 523, as also illustrated in FIG. 5C. 
The selection of project team members is based on project requirements. Other 
elements in the organization of a project team are the finalization of the project team's 
organization structure and documentation in the organization chart of the project plan. 

25 The organization should further update a training needs matrix to document (1) the 

training required of each project team member and (2) the proposed means for 
fiilfiUing the training. This document is used to track project team member training. 
In another implementation, organizing a project team in step 523 furliier requires the 
organization to determine, as a team, the project's mission, vision, and charter, and 

30 then to document these determinations in a project plan and orientation binder that is 
created as required for the CMM. 
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Returning to FIG. 5C, another task in the organization of project resources in 
step 520 is to estabhsh other resources, step 524. Specifically, the organization 
performs this task to organize the physical resources, such as hardware or software, 
provided by program management and to develop the orientation and/or training 
5 needed to support the activities of the project team. The establishment of other 

resources in step 524 helps create a work environment that promotes communication, 
collaboration, and group cohesion. 

Also as illustrated in FIG. 5C, the organization of project resources in process 
520 further includes enabling resources, step 525. Organizations perform this step to 

1 0 orient and train team members, manage the physical resources assigned to the project, 

and coach and evaluate team members. The enabling of resources in step 525 aids the 
project manager in motivating and challenging team members and while helping to 
ensure that various project personnel beHeve their work to be important. Specifically, 
the organization should commimicate the project's mission, vision, and charter to new 

1 5 team members. Large projects may also elect to formalize these items at the program 
level, and projects may conduct one or more meetings that include all team workers. 

As illustrated in FIG. 5D, another step in the project management 500 is to 
control project work, step 530. In step 530, project management monitors the 
execution of the project against project plans and makes adjustments as necessary. 

20 Project Status Reports are prepared for the Project Sponsor. Potential and actual 
problems are identified through the measuring and monitoring of progress and 
performance against the Project Plan. Depending on the type of problem identified, 
an Issue, Risk, SIR or CR is logged. Project management is expected to take 
appropriate corrective actions to resolve problems that are discovered. Step 530 and 

25 constituting tasks 53 1-37 closely correlate, respectively, to steps 230-37, described in 
FIG. 2D and its accompanying text. The organization performs step 530 to control 
project execution throughout the project's life cycle. The control of project work in 
step 530 includes identifying potential and actual problems by monitoring and 
measuring progress against the project plan. 

30 As illustrated in FIG. 5D, the controlling of project work in step 530 begins 

with the releasing of work packages, step 531. To release work packages, the 
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organization should assemble and release work packages according to the work plan, 
and communicate their requirements to the assigned team members. Work packages 
are generally described in the CMM criteria and generally relate to the task and 
functions given to the various workers in a project. The project team then performs 
5 the work needed to develop the required dehverable good. During step 53 1, the 

organization preferably acts to ensure that each team member understands assigned 
responsibilities, including target dates and budgets. Furthermore, the organization 
should encourage each team member to provide input regarding various assigned 
responsibilities, including target dates and budgets, and to accept and carry out these 

1 0 assigned responsibilities . 

As depicted in FIG. 5D, a following step in the control project work, step 530, 
is measuring performance, step 532. The measuring of performance in step 532 
generally includes capturing actual results and calculation of metrics in order to 
manage performance. Capture metrics, as outlined in the organization metrics plan 

15 formedinstep510,includecost, effort, scope, quality, and schedule. The 

organization should further track project infrastructure/ technical requirements, such 
as hardware, software, and performance requirements, that were outlined during 
planning in step 510. The organization should also analyze any deviations from the 
project plan and identify, in a timely manner, the causes for the deviations. 

20 Concurrent with the measuring of performance in step 532 is managing 

performance, step 533, as illustrated in FIG. 5D. Managing performance in step 533 
generally requires tiie organization to manage project performance against the 
previously defined project and work plans. To project performance in view of the 
project and work plans, the organization proactively assesses performance, status, 

25 quality, and risk. When the actual results from the development of the project do not 

match the plans, the organization should fiuther determine alternative goals or 
actions. The implementing organization may further obtain approval for corrective 
actions, and then take corrective actions. The corrective actions may include, but are 
not limited to, work process changes, team building, training, increased or decreased 

30 supervision, work assignment changes, reassignment of team members, initiation of 

risk responses, the change of requests to be pursued with program management as part 
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of the configuration management process, project replanning changes that specify 
needed modifications to the project plan, project plan revisions (work package 
changes, etc.) or escalation to program management. The organization should also 
reevaluate project decisions throughout the project life cycle, when project triggers or 
5 other issues, risks, etc. arise. In step 533, the organization may also manage team 

member performance according to organizational standards and tools. 

Continuing with FIG. 5D, following the measuring of performance in step 532 
and the managing of performance in step 533, the organization communicates project 
status, step 534. During the step 534, the organization generally develops and 

10 communicates project status to all project stakeholders according to the Project Plan. 

The project stakeholders include project and senior management and other affected 
groups. The organization further conducts status and review meetings involving 
affected groups as appropriate. During the communication of project status in 
step 534, the organization should document meeting minutes as required for the 

15 CMM. 

Continuing with FIG. 5D, following the communication of project status in 
step 534, the organization obtains acceptance of interim deliverable goods, step 535. 
Obtaining acceptance of interim dehverable goods in step 535 generally requires that 
the organization obtain acceptance of interim dehverables by all designated 
20 stakeholders, as appropriate, at key interim points throughout the project life cycle. 

Any acceptance of final deliverables takes place in connection with completing the 
program. 

Another task in the control of project work in step 530 is to execute project 
management processes, step 536. The organization should execute these processes in 

25 conjunction with other project control activities, such as measurement activities and 

status reporting. Also, the project management processes may occur continuously, 
periodically, or may be event driven. One project management process in step 536 is 
risk management, which addresses the identification, analysis, and avoidance/ 
mitigation aspects of risk management on a project. One project management process 

30 is risk identification, during which the organization identifies, names, and describes 

the various risks. The organization should further generate a list of specific 
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incremental risks in the project's risk-tracking tool. The organization documents 
known triggers for a risk, the potential damage for each risk item, and references for 
the sources of risk. Another risk management task in step 536 is risk analysis, in 
which the organization analyzes the identified risks. In risk analysis, the organization 
5 should classify the risks and include any additional information necessary to support 

the analysis. The organization may then select a rank/prioritized list of top risks. For 
instance, the organization may create a list of the top five risks to a project. Another 
risk management task is risk avoidance and mitigation. Risk avoidance activities 
address the sources of a risk, thereby reducing the probability that it should become a 

1 0 problem. For a top-ranked/prioritized risk, the organization should identify how the 

risk can be avoided. Risk mitigation measures attack the consequences of a risk, 
reducing the risk's potential impact on the project. For the top-ranked/prioritized 
risks, the organization may identify actions to reduce the impact of the risk if it 
occurs. The organization may also use DAR to assess the risks. 

1 5 Another proj ect management process in the execution of proj ect management 

processes in step 536 is scope management, which addresses the acceptance of 
requirements to define scope and the requirements of change control process. For 
instance, one scope management task is requirements development. During the task 
of requirements development, the organization identifies and documents requirements 

20 needed to promote and ensure bi-directional traceability, so that the organization may 

trace requirements between the development and the testing of the requirements. As 
with all work products, requirements are preferably placed under configuration 
management (CM), as defined in the CMMI. Another scope management task is 
requirements acceptance, during which the organization documents and reviews 

25 requirements with all affected groups and obtains acceptance from the affected 

stakeholders. The organization should further establish baseUne standards for 
satisfying the requirements. Another scope management task for the organization is 
to make any required changes to the requirements and their baselines. The 
organization generally follows the project's change control process for any changes to 

30 baselined requirements. Specifically, the organization submits a change request; 

reviews a change request; performs impact analysis, including cost, schedule, and 
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efforts impacts; determines disposition; implements change, including associated 
impact to other work products and activities; and notifies requester and affected 
groups. Again, the organization may determine if it is necessary to use DAR to assess 
changes in scope. 

5 Another project management process in step 536 in the execution of the 

project management processes is configuration management. This task addresses the 
set of activities performed to establish and maintain the integrity of the project work 
products throughout the project's Ufe cycle. One set of configuration management 
tasks relates to configuration identification activities. During the configuration 

10 identification activities, the organization identifies, names, and describes each of the 
configuration items that should be placed under configuration management. In 
particular, all work products should be placed imder some type of configuration 
management. During the configuration identification activities, the organization 
generally uses the CM plan to define a baseline for the configviration items and to 

1 5 indicate the level of configuration management for each item. Another configuration 
management process in step 536 is configuration of control activities. Generally, the 
organization requests, evaluates, approves or disapproves, and implements changes to 
the baselined configuration items defined during the configuration identification 
activities. All of the configuration items should be archived and placed under the 

20 project's documented change control process. Configuration of status accounting 

activities is anotiier configuration management process in step 536. During this 
process, the organization records and reports the status of the project's configuration 
items using a configuration management status report. Similarly, the organization 
should fiirther perform configuration audits. Specifically, the organization may, using 

25 the CM plan, determine the extent to which actual configuration items reflect the 

planned configuration items. The purpose of this task is to ensure that the entire 
configuration is correct and complete. The organization should fiirther document 
results as required in the CMMI, using a configuration audit. 

Another project management process of the execution of the project 

30 management process in step 536 is issue management and escalation. This task 

involves the identification and documentation of issues using an issue tracking tool, as 
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well as a review of the issue and an analysis of any impact on deliverables, scope, 
contingency, resources, costs, schedule, and/or quaHty. Specifically, the organization 
should identify a resolution approval party, an issue's owner, and determine expected 
time fi:^es. The organization may also determine if it is necessary to use DAR to 
5 assess the issue, as described above. The organization may further research and 

identify issue solution alternatives. Subsequently, the organization may refer the 
issue to program/ senior management when: (1) the project manager cannot resolve 
the issue internally, (2) when the issue impedes the progress of a project, and when 
the issue is beyond the authority of the project manager to resolve. These are 

10 generally issues that (1) cannot be resolved within a project team, (2) are resolvable 

with action items, (3) can be escalated to the next level, (4) are reactively discovered 
during the course of development, (5) affect program/project scope, costs, schedule, 
projected business performance, or high-level design, (6) affect multiple projects or 
releases, and/or (7) involve groups outside the project that affect project delivery. 

1 5 The organization should accordingly monitor issues status while approving or 
rejecting resolutions. At the same time, the organization should communicate 
resolutions to stakeholders and affected parties and take corrective action as described 
above in the context related to management of performance tasks. 

Returning to FIG. 5D, another step during the process of controlling the 

20 project work in step 530 is to update the project plan and subordinate plans, step 537. 

In particular, throughout the life cycle of the project, the project plan and subordinate 
plans (Risk Management, Configuration Management, Work Plan, Subcontractor 
Management Plan, Community, and Sponsorship Plan) should be updated as 
appropriate, by the organization to reflect any changes on the project that should 

25 affect the content of the documentation. 

Another step of project management 500 is to complete the project, step 540, 
as illustrated in FIG. 5E. In step 540, project closeout is performed and overall 
project results are evaluated. Project Management verifies that all activities for a 
project are complete so that all resources can be released and all documentation and 

30 responsibilities can be transferred to the necessary parties. In this way, step 540 

enables Project Management and the Project Sponsor to measure the success of the 
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project and use results of the project as inputs to future efforts. A first step in 
completing the project in step 540 is to obtain a formal acceptance of deUverable(s), 
step 541, and this task 541 entails obtaining sign-offs on the final deliverables from 
the appropriate stakeholders. In effect, each stakeholder should agree that the project 
5 is, in fact, complete. Another step in completing the project during step 540 is the 

preparation of final documentation, step 542, in which the organization completes 
final revisions and packaging of deliverables. Likewise, the organization furthers the 
completion of the project in step 540 by transferring responsibility for deliverables, 
step 543, to formally transition responsibility for the deliverables to the appropriate 
10 parties. The transfer of responsibility for deliverables in step 543 generally includes 

the transition of training materials, operations manuals, and other supporting 
documents. 

Continuing with the completion of the project in step 540, the organization 
evaluates the project, in step 544, by assessing the success of the project, summarizing 

15 the project's accomplishments, discussing/documenting any items for improvement, 
and channeUng the resulting information through the appropriate quality management 
process. The various results of the evaluation of the projects in step 544 should be 
recorded in a closing memo, as specified in the CMMI. The results of the evaluation 
may include (1) reviewing the project work plan ; (2) updating the estimates; 

20 (3) sending the project's actual results to the owners of the estimating tool; 

(4) submitting final project metrics to the Soflware Engineering Process Group 
(SEPG); and (5) conducting an SQA debriefing to discuss results of the SQA program 
and also process improvement points. Another step in the completion of the project, 
step 540, is to release resources, step 545. The organization performs step 545, for 

25 instance, to "roll off human resources from the project and to return equipment and 
supplies to the appropriate custodian, thereby freeing these resources for use on other 
projects. 

Returning to FIG. 5 A, the next task in the project management 500 is software 
quality assurance (SQA) review execution, step 550, the substeps of which are 
30 illustrated in FIG. 5F. During the SQA review execution of step 550, the organization 

may conduct software process and work product quality assurance reviews to verify 
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project adherence to standards and procediires. The first step of the SQA review 
execution 550 is to complete a project plan and metrics workbook, step 551. In this 
way, the project manager and SEPG liaison are encouraged and required to identify 
deliverables and processes to be reviewed; ensure that deliverables in the Project Plan 
5 and Work Plan are consistent; identify reviewers, reviewees, and review criteria; 
identify roles and responsibilities; identify SQA metrics; complete the quaUty 
program section of the project plan; and update the metrics workbook with the SQA 
review schedule. 

Another step of the SQA review execution 550 is to prepare for an SQA 

10 review, step 552. In the SQA review, the project manager provides job accounting 

information to the SQA reviewer and sets SQA review expectations. In preparation 
for the SQA review during step 552, a deliverable owner (i.e., a party responsible for 
producing a deliverable product or service) provides the deliverable to be reviewed, 
where "deliverable" is defined in the CMMI. The deliverable owner further provides 

1 5 contact and availability information to the SQA reviewer and provides review criteria 

and standards to the SQA reviewer. In response, the SQA reviewer gathers the 
deliverable product or service, reviews the proposed review criteria and standards, 
schedules a meeting with the deliverable owner, and receives job accoimting 
information JQrom the project manager. 

20 Returning to FIG. 5F, another step of the SQA review execution 550 is to 

conduct the SQA review, step 553. During the SQA review in step 553, the SQA 
reviewer generally reviews deliverables against review criteria/standards, identifies 
nonconformance items, and follows up with the deliverable owner as needed to meet 
the requirements of the CMMI. At the same time, a SEPG liaison reviews the project 

25 management deliverables against a best practices matrix. For the CMMI, the 

deliverable owner should be able to continue answering any questions. 

Another step of the SQA review execution 550 illustrated in FIG. 5F is to 
prepare the SQA report, step 554. The SQA reviewer prepares a detailed summary of 
findings and recommendations, including item number, date reported, and an accurate 

30 description of nonconformance items. The SQA reviewer then distributes the SQA 
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report to the deliverable owner and the SEPG haison. The deliverable owner should 
then document responses in the SQA report template. 

Continuing with FIG. 5F, another step of the SQA review execution 550 is to 
discuss nonconformance items, step 555. Specifically, the organization should 
5 require the deliverable owner to discuss any nonconformance items with the SQA 

reviewer. In addition, the deliverable owner updates the SQA Report with proposed 
resolution(s) and projected completion date(s) for agreed upon items. The deliverable 
owner also escalates disagreement items for faciUtation and updates a return report, as 
well as any necessary documents to the SQA reviewer for verification. In response, 

10 the SQA reviewer should discuss nonconformance items with the deliverable owner 

and verify the resolution of nonconformance items. During step 555, the SEPG 
haison and the project management should also resolve escalated nonconformance 
items and resolve, on a case-by-case basis, any issues that may arise due to scheduling 
conflicts between the SQA reviewers and the deUverable owners. 

1 5 Continuing with FIG. 5F, another step of the SQA review execution 550 is to 

track SQA metrics, step 556. In step 556, the SQA reviewer sends the final report to 
the deliverable owner and the SEPG Haison. At this point, the SEPG Uaison may 
update an SQA tracking tool and forward the final report and metrics to the project 
sponsor and project manager. Typically, the SEPG haison includes metrics such as 

20 the SQA schedule variance; the number of nonconformance items; the cost/savings of 

the SQA program; the value added by conducting SQA reviews; and a best practices 
percentage showing compliance with desired best practice policies. As required by 
the CMMI, the project manager keeps copies of documentation and reports. 
Another aspect in the project management 500 is suppher agreement 

25 management, step 560, which is generally illustrated in FIG. 5G. The suppHer 

agreement management 560 comprises subcontractor management in step 560(a) and 
product acquisition in step 560(b). Specifically, the subcontractor management in 
step 560(a) comprises the tasks of planning subcontractor management, step 561; 
organizing subcontractor management resources, step 562; controlling subcontractor 

30 management, step 563; and completing subcontractor management, step 564, as 

illustrated in FIG 5G. Likewise, product acquisition in step 560(b) comprises the 
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tasks of planning product acquisition, step 565; organizing product acquisition, 
step 566; controlling product acquisition, step 567; and completing product 
acquisition, step 568, as depicted in FIG 5G. 

FIG. 5H depicts that tasks 561(a)-561(f) comprise the planning of 
5 subcontractor management in step 561. In step 561, proj ect management plans for the 

project's use of subcontractors including developing criteria to be used for 
subcontractor selection. The first task in step 561 is to identify the need for a 
subcontractor, step 561(a). In step 561(a), the organization identifies a need for a 
subcontractor. Before the need for a subcontractor is determined, the business 

10 requirements for the project should be defined. The objective is to describe "what 

needs to be done and/or achieved" and which development team/s should be 
instrumental in implementing this requirement. The supporting analysis and research 
provide input with regard to the requirements, including the current capability 
analysis, constraint analysis, best practice research, and potential delivery options. If 

15 the project team does not have the resources to satisfy these requirements, then a 

subcontractor should be considered. Again, the organization may use DAR if 
necessary to evaluate the need for a subcontractor. If a subcontractor is needed, the 
organization should update the supplier agreement management section of the project 
plan with a description of the subcontractor arrangement. The organization may then 

20 prepare the subcontractor management plan. 

Returning to FIG. 5H, the organization's next action during the planning of 
subcontractor management in step 561 is to define a subcontractor statement of work 
(SOW), step 561(b). The subcontractor SOW should clearly define the scope and 
objectives of the subcontract, the process that should be used to manage the 

25 subcontractor, and any standard contract clauses. The SOW should also provide as 

much detail as possible about the planned subcontract, including the contract 
monitoring process, the quality management process, the configuration management 
process, and the contract closure process. A proposal/project team is generally 
responsible for identifying the technical requirements that the subcontractor shotdd 

30 satisfy. 
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As depicted in FIG. 5H, the organization's next action during the planning of 
subcontractor management in step 561 is to develop subcontractor selection criteria, 
step 561(c). Prior to assessing subcontractors, the organization should define the 
selection criteria. Whereas some criteria should be generic, such as quality, service, 
5 value, and past performance, there is greater value in defining specific criteria that 

apply to different categories of assets and services to be procured, especially those 
criteria concerning longer-term cost considerations. Selection criteria should also 
reflect defined business needs. To satisfy this step 561(c), the proposal/project team 
should create the subcontractor selection criteria using the template provided. 
10 Next, in step 561(d), the organization should develop a subcontract pricing 

H mode. In general, after defining the statement of work, it is necessary to establish the 

p type of contract that will be used for the subcontract. It is important to determine the 

1 i type of contract early in the process, as it has a fimdamental impact on the 

=J subcontractor's proposal and economics of the program. This work should be closely 

m 

; g 1 5 coordinated with the development of the contract strategy. 

Returning to FIG. 5H, the organization's next action during the planning of 
i U subcontractor management in step 561 is to create a subcontractor long list, 

i.j step 561(e). Using the subcontractor selection criteria template provided, the 

;=f organization in step 561(e) identifies the long hst of subcontractors that will be 

20 invited to propose. This Ust may be based on the following criteria: satisfaction with 

existing/previous subcontractor work, market share of the subcontractor, industry 
reputation of the subcontractor, proximity of the subcontractor, availability of the 
subcontractor, financial status of the subcontractor, etc. 

As depicted in FIG. 5H is the prepare/finalize request for proposal (RFP), 
25 step 561(f). The RFP should be created m step 561(f) after the need for a 

subcontractor has been established in step 561(a), the statement of work has been 
defined in step 561(b), the selection criteria have been established in step 561(c), the 
pricing model has been estabhshed in step 561(d), and the appropriate terms and 
conditions have been established. The RFP should be finalized with input fi-om all 
30 relevant stakeholders. 
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As depicted in FIG. 5G, the next task in the suppUer agreement management 
in step 560(a) is to organize subcontractor management resources, step 562. The 
organization performs step 562 to organize resources associated with subcontract 
management. In step 562, the project Work Plan is updated to account for 
5 subcontractors. Tasks that will use subcontractor resources are docvimented. 

Subcontractor Selection Criteria are finalized and a subcontractor is selected. Turning 
now to FIG. 51, the organization of subcontractor management resources in step 562 
comprises the tasks of developing work breakdown structure (WBS) and a resource- 
loaded work plan, step 562(a); finalize subcontractor selection criteria, step 562(b); 

10 issue a request for proposal (RFP) , step 562(c); receiving bids, step 562(d); 

evaluating bids to select a suitable subcontractor, step 562(e); and negotiating and 
finalizing a subcontract, step 562(f). It should be noted that steps 562(a)-(e) in the 
flow chart in FIG. 51 represent the potential tasks that would be completed to select a 
subcontractor, but many of these steps may be omitted based on project requirements. 

15 In the development of the WBS and resource-loaded work plan of step 562(a), 

the WBS decomposes each business capability into manageable units and depicts the 
total scope of the solution needed to achieve the program/project objectives. The 
work plan sets out the major work processes and constituent units of work that will be 
used to accomphsh the project. The resovirce-loaded work plan then matches 

20 available resources with each task in the work plan. Both the WBS and the resource- 

loaded work plan should document the tasks that will be completed using 
subcontractor resources. 

. In step 562(b), the organization should finalize subcontractor selection criteria, 
as depicted in FIG. 51. In step 562(b), the organization updates the subcontractor 

25 selection criteria established during the plan subcontractor maiagement of step 561 to 

finalize the criteria that will be used to evaluate subcontractor proposals. Continuing 
with FIG. 51, during step 562(c), the organization next issues an RFP and distributes 
the RFP to a Ust of subcontractors identified for solicitation in step 561(e). The 
organization then receives bids, step 562(d), to gather proposals from subcontractors. 

30 The organization should then evaluate the bids and select a suitable 

subcontractor, step 562(e) in FIG. 51. In particular, as bids are received firom 
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subcontractors, the responses should be entered into a subcontractor selection criteria 
matrix to facilitate the evaluation process. Evaluators should also review the potential 
risks associated with each subcontractor. Once all responses have been entered into 
the matrix and all potential risks have been assessed, a selection can be made by the 
5 organization. 

The organization should next negotiate and finalize a subcontract, step 562(f) 
in FIG. 51. After the subcontractor is selected, it may be necessary to make additional 
negotiations to finalize the contract. As a result of finalizing the subcontract, it may be 
necessary to update the project plan and/or subcontractor management plan with any 

1 0 new conditions, such as the need to provide project-fiimished facihties. 

Returning to FIG. 5G, the next step in the subcontractor management in 
step 560(a) is to control subcontractor management in step 563. In step 563, the 
organization acts during project execution to monitor and control subcontractor 
activities for subcontractors that do not fimction as part of the project team. 

1 5 Subcontractors that work as part of the project team follow the processes outlined in 

the step of control project work in step 530. In addition, there should be regular status 
meetings with the subcontractor. During step 563, the work and work products of 
subcontractors are monitored through visual observation and/or Subcontractor Status 
Reports. Corrective action is taken as problems arise. 

20 Substeps 563(a)-(b) of the control subcontractor management in step 563 are 

depicted in FIG. 5J. Specifically, in step 563(a), the organization monitor 
subcontractor performance: The project manager or designated team member 
overseeing the subcontractor should observe the subcontractor's performance on a 
regular basis and manage all communications with the subcontractor. If the 

25 subcontractor fails to perform as expected (e.g., late delivery, poor quality, etc.), the 

organization should act to remedy these failures to minimize their harmful effects on 
the project. 

Likewise, in step 563(b), the orgmiization should receive subcontractor 
reports, as illustrated in FIG. 5J. The subcontractor should submit all reports to the 
30 project team as specified in the subcontract. This may include status reports, turn- 

around documents, invoices, metrics, etc. These reports should be used to track 
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subcontractor performance against the work plan and schedule milestones and 
evaluate quality of work. 

Returning to FIG. 5G, the final step in subcontractor management in 
step 560(a) is to complete subcontractor management, step 564. In step 564, project 
5 management verifies that the subcontractor has completed all tasks outlined in the 

subcontract and that technical performance requirements are satisfied. If the 
subcontractor successfiilly satisfies all contract requirements, both administrative and 
technical, the contract close out process occurs. If not, project management takes 
corrective action. Project Management updates the Closing Memo based on 

10 subcontractor deliverables and performance as necessary. As depicted in FIG. 5K, the 
tasks in the completion of subcontractor management in step 564 include the 
determination of whether contract requirements are satisfied, step 564(a); determining 
if technical performance requirements are satisfied, step 564(b); transitioning 
responsibilities and work products, step 564(c); and closing contract, step 564(d). In 

15 determining if contract requirements are satisfied in step 564(a), the organization 

assesses whether the subcontractor has failed to satisfy the contractual requirements. 
The organization further determines if any corrective actions may be needed. 

Continuing with FIG. 5K, in determining if technical performance 
requirements are satisfied in step 564(b), the project manager or designated team 

20 member oversees a subcontractor and is responsible for assessing the technical 

performance of that subcontractor. The acceptance criteria for contractual closeout 
are docimiented in the SOW and should be used to evaluate the subcontractor's 
performance. This assessment may include a review of deliverables, metrics, 
invoices, etc., submitted by the subcontractor. If the subcontractor fails to satisfy the 

25 technical performance requirements of the contract, corrective action may be needed. 

Referring again to FIG. 5K, the next task in completing the subcontractor 
management is the transition of responsibilities and work products, step 564(c): Once 
the subcontractor has successfully completed all work stated in the contract, it is 
necessary to transition the responsibilities and work products of the subcontractor to 

30 the appropriate party. Step 564(c) may require the subcontractor to train persoimel in 

a given area, hand over system documentation and manuals, etc. 
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Then, in step 564(d), the organization may close the contract with the 
subcontractor, as illustrated in FIG. 5K. If the subcontractor successfiiUy satisfies 
both administrative and technical contract requirements, the contract closeout process 
can occur. The contract closeout process may include the collection of information, 
5 such as performMice metrics, from the subcontractor if this requirement was specified 

in the statement of work, request for proposal, or contract. 

Returning to FIG. 5G, the corollary to the subcontractor management of 
step 560(a) is the product acquisition of step 560(b). The first task in the product 
acquisition is to plan the product acquisition in step 565. The organization performs 
10 step 565 to plan acti\dties related to product selection and implementation. In step 

y= 565, the project's product needs are identified. The project's detailed approach to 

J={ product acquisition is outlined in the Product Selection Approach. After determining 

□ a need for a product exists, a high-level review of the market is conducted to 

%j determine possible vendors and the product selection criteria are developed. It should 

%l 1 5 also be noted that, in some cases, there are outside factors that govern the selection of 

f products. Therefore, the following tasks 565(a)-565(d) may not always be necessary 

fU or inclusive. In addition, these tasks are only necessary when the product will be 

Jrl turned over to the client. 

P Turning now to FIG. 5L, the first task in the planning of product acqmsition in 

20 step 565 is to identify a need for a product, step 565(a). In step 565(a), the 

organization determines if business needs can be satisfied with the implementation of 
an off-the-shelf product. Step 565(a) may also involve participation from the 
proposal/project team and the client. The organization may generally follow the 
guideKnes in a DAR Reference document. Specifically, the triggers and thresholds 
25 docimiented in the project plan determine if it is appropriate to use DAR to evaluate 

the need and/or selection of an off-the-shelf product. The organization may Hkewise 
use the project plan to describe the project's need for an off-the-shelf product and to 
identify the areas in which it may be necessary or desirable to use an off-the-shelf 
product. 

30 As depicted in FIG. 5L, the next task in the planning product acquisition in 

step 565 is to develop a product selection approach, step 565(b). In step 565(b), the 
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organization may document the project's detailed approach for product acquisition. 
Likewise, another task in the product acqmsition 560(b) is to survey potential product 
providers, step 565(c). In step 565(c), the organization may conduct a high-level 
review of the market to determine potential product providers or contact an alliances 
5 group for assistance in identifying providers. The organization may then document 

potential providers on a vendor long list according to product selection criteria. This 
step 565(c) may include input from several different sources such as Mtemet research, 
participants with past project experience, independent product ratings, etc. In some 
cases, step 565(c) may not apply as the project sponsor may dictate the specific 

1 0 application to be used. 

Continuing with FIG. 5L, the next task in the planning of product acquisition 
in step 565 is to finalize the list the product providers to be invited to propose, 
step 565(d). In step 565(d), the organization identifies a hst of product providers to 
be considered based on the information gathered during the survey of potential 

15 candidates. The organization may select providers that satisfy most of the project's 

requirements. In step 565(d), the organization may refer to predetermined product 
selection criteria with the product providers to be considered. 

Continuing with FIG. 5G, the next task in the product acquisition step 560(b) 
is to organize product acquisition, step 566. In step 566, the organization organizes 

20 resources associated with product acquisition. Furthermore, the product selection 

criteria are finalized and vendors are invited to demonstrate their products. At the end 
of this task, final product selections are made. It should be noted that, in some cases, 
there are outside factors that govern the selection of products, and, therefore, some or 
all of steps 565(a)-(e), described below in FIG. 5M, may be skipped as necessary. 

25 Turning now to FIG. 5M, the individual tasks that comprise the organization 

of the product acquisition m step 566. The first task is to finalize product selection 
criteria, step 566(a). In step 566(a), the organization should develop finalized 
selection criteria based on the organization's economic needs and goals, such as costs, 
timeframe, and quahty concerns. 

30 Next, in step 566(b), the organization may define business scenarios, as 

illustrated in FIG. 5M. Using the product selection criteria formed in FIG. 566(a), the 
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organization may develop business scenarios that may then be used to form a 
questionnaire. In this way, the business scenarios may be used to evaluate product fit 
and performance during vendor demonstrations. 

The next task in FIG. 5M is to prepare and distribute a request for proposal 
5 (RFP), step 566(c). In step 566(c), the organization should develop a RFP that 

requires the vendors and providers to propose similar configurations and have all 
hardware, software, and on-site consulting services (in days) identified and itemized. 
Providers can also submit their idea of an optimal configuration. Furthermore, the 
RFP should include as much information about application interaction as possible. 
1 0 Returning to FIG. 5M, another task in the organization of product acquisition 

in step 566 is to conduct vendor demonstrations, step 566(d). In step 566(d), each 
finalist should provide a product demonstration. During the demonstration, the 
organization should evaluate how well each provider/vendor meets the various 
business scenarios. 

1 5 Next, the organization may compare costs and benefits of product providers, 

step 566(e), as illustrated in FIG. 5M. In particular, the organization may use the 
product selection criteria to compare the proposals submitted by the product provider 
finahsts. Also evaluate the potential risks associated with each product. 

Continuing with FIG. 5M, another step in the organization of the product 

20 acquisition is to make a final product selection, step 566(f). The orgMiization may 
select the product provider based on the final scores in the product selection criteria 
and an assessment of potential risks. Step 566(f) may also includes the preparation of 
a purchase agreement or contract. It may also be necessary in step 566(f) to update 
the project plan to document any new conditions that result from the product 

25 acquisition, such as the need to provide project-furnished facilities. 

Returning to FIG. 5G, the next step in the product acquisition of step 560(b) is 
to control product acquisition, step 567. In step 567, the selected product is installed, 
testing is performed, and the performance of the product is evaluated. The 
organization may perform step 567 after acquiring the product to ensure that the 

30 product satisfies business needs and performs as anticipated. It should be noted that 

these tasks are typically performed only for new products that have not been 
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previously tested and implemented. For products that have been previously 
implemented, application testing and performance are evaluated during previous 
product and acceptance testing. 

The substeps in the control of product acquisition in step 567 are depicted in 
5 FIG. 5N. The first task in controlling product acquisition in step 567 is to install or 

otherwise use the product in the environment to be used for acceptance and 
performance testing, step 567(a). 

Next, in step 567(b), the organization conducts application testing, as 
illustrated in FIG. 5N. Specifically, once the product has been delivered, it is 

10 preferable in step 567(b) to perform a fit analysis to ensure that the software satisfies 
the business scenarios as originally intended. The fit analysis should map the product 
characteristics against both the existing user service class characteristics and the 
existing underlying components of the delivery vehicles (execution, operations, 
development, computing platforms, and network architecture). 

1 5 Continuing with FIG. 5N, the next task in the control of the product 

acquisition is to evaluate application performance, step 567(c). Three different 
methods are available to evaluate product performance in step 567(c): comparison, 
application sizing, and electronic spreadsheet analysis. Comparison analysis may be 
performed using existing installations of the product with similar environments, 

20 operations, and configurations. Some product vendors perform application sizing to 
determine if the product is adequate for the project needs, but results should be 
interpreted with caution. Electronic spreadsheet analysis translates business 
transactions into resource utilization and service times for evaluation. The accuracy 
of electronic spreadsheet analysis is driven primarily by the knowledge of business 

25 functions, transaction rates, and package architecture. 

Returning to FIG. 5G, another step in the product acquisition is to complete 
the product acquisition, step 568, to close out the product acquisition tasks. In step 
568, project management determines if the contract requirements are satisfied. Once 
the product has been assessed and meets all performance and functional needs, the 

30 product and the job responsibilities associated with the product are transitioned to the 

appropriate party. At this point, the contract with the vendor is closed. The tasks in 
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the completion of the product acquisition in step 568 are illustrated in FIG. 50. 
Specifically, the completion of the product acquisition in step 568 comprises the tasks 
of determining if contract requirements are satisfied, step 568(a); determining if 
performance issues require contract to be closed step 568(b); transitioning the 
acquired product, step 568(c); and closing the product acquisition contract, 
step 568(d). In the determination of whether purchase contract requirements are 
satisfied during step 568(a), the organization assesses the product against the contract 
requirements to determine if the agreed upon conditions have been met. Next, in 
step 568(b), the organization determines whether performance issues require the 
product purchase contract to be closed. Specifically, the organization decides if the 
product meets performance requirements. If the product fails to meet performance 
requirements, it may be necessary to terminate the contract. Contact the alUances 
group for assistance in identifying a product with better fit or performance. 

Once the product has been assessed and meets all performance said functional 
needs, it is necessary to transition the product and the job responsibilities associated 
with the product to the appropriate party. Accordingly, in step 568(c), the 
organization may transition the product as needed, as illustrated in FIG. 50. 
Step 568(c) may require the organization to train the appropriate party, handing over 
system documentation and manuals, etc. Then in step 568(d), the organization may 
close the purchase contract after verifying that contract requirements have been 
satisfied. 

Delivery Management 

Returning to FIG. 1, the next step of the CMM in a BOX method 10 of the 
present invention is to implement dehvery management 600. Delivery management 
600 relates to the activities undertaken to develop a system software application for 
eventual delivery to chents. The Delivery management step 600 translates the 
required business outcomes into a business solution. A business solution is the 
combination of business process, a technology solution and organizational changes 
that collectively create value by improving business performance. The Dehvery 
Management Module defines a multi-functional approach for taking each business 
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solution from analysis to dqjloyment. As depicted in FIG. 6A, the delivery 
management 600 encompasses four stages of work: analysis, step 700; design, 
step 800; building and testing, step 900; and deployment, step 1000. One of the 
delivery programs should be mobilized for each business solution to be delivered. 
5 In analysis stage 700, the organization accesses and defines the tasks to be 

accomplished for delivery of the desired products. During stage 700, business, user 
and interface requirements are defined as necessary to define and commit to a specific 
implementation and release plan. The information gathered during stage 700 focuses 
on business requirements, describing it to the level of detail needed to finalize the 

10 delivery releases, define the specific requirements, and resolve implementation issues. 

As illustrated in FIG. 7A, the analysis stage 700 consists of defining a business case, 
step 710; gathering and analysis of requirements, step 720; assessment of the 
deployment environment, step 730; and identification and analysis of the apphcation 
and interface requirements, step 740. 

15 The first step in the analysis stage 700 is the defining of a business case, 

step 710, which is illustrated in FIG. 7B. In step 710, the organization defines the 
business case to determine benefits to be derived from, and justification for a 
proposed business solution. When defining a business case in step 710, the 
organization first determines an economic evaluation approach, step 711. 

20 Specifically, the organization performs this task to obtain commitments from the 

appropriate stakeholders in the sponsoring organization on the overall implementation 
approach for the proposed solution. This task freats the process of implementing a 
solution as an investment. 

The organization subsequently creates a model structure, step 712. During 

25 this task, the organization obtains an agreement from the sponsoring organization 
regarding the structure of the model used to determine the benefits of implementing 
the proposed solution. For example, benefits to be derived may be expressed in terms 
of increased market share or reduced operating costs. 

The organization next forecasts baseline business performance, step 713, to 

30 determine how the business should perform without the proposed solution. The next 

step in the analysis stage 710 is to project net change journey benefits, step 714, 
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during which the organization attempts to predict and quantify the benefits that the 
sponsoring organization should derive from implementing the proposed solution. A 
further step in the analysis stage 710 is to assemble a business case, step 715. During 
step 715, the organization documents a rationale for implementing the proposed 
5 solution. Ultimately, this document should serve as a motivational tool for change. 

Turning now to FIG. 7C, the next step in the analysis stage 700 is the 
gathering and analysis of requirements, step 720. In step 720, the current business 
environment is assessed and new requirements for the business and users are defined. 
During the gathering and analysis of requirements in step 720, the organization 

10 analyzes its current business, step 722, to obtain an accurate picture of its elements, its 

operation, and its performance. The organization then identifies user and business 
requirements, step 724, to define and document high-level requirements for desired 
solutions. These requirements include changes to human performance, business 
process, and technology. The organization shoidd seek to ensure that these 

15 requirements meet the needs stated in the proposal, business needs statement, work 
request, or task order, including interfaces to other systems. During step 724, project 
participants impacted by the requirements should be involved in the review and sign- 
off of the requirements. Another step in the gathering and analysis of requirements in 
step 720 is to document the new business environment, step 726. In step 726, the 

20 organization documents user locations and transaction volumes in any new business 
environment. 

As illustrated in FIG. 7D, the analysis stage 700 continues with the assessment 
of the deplo)ment environment, step 730, to ensure that deployment concerns and 
needs are considered sufficiently early in the development process. The objectives of 

25 the task are to consider the geographical, infirastmcture, operational, and cultural 
differences between the current structure of the sponsoring organization and the 
desired structure, to define the deployment requirements, and to determine the optimal 
deployment mechanism. 

In the evaluation of the deployment environment in step 730, the organization 

30 ^sesses its release approach, step 732, to review the deployment plan, particularly to 

identify risks and to justify costs for deployment. The organization further identifies 
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deployment requirements, step 734, to identify deployment requirements for the 
proposed solution. A key deployment requirement is the production and release of the 
deliverable product or service. The organization should document the identified 
deployment requirements within a business requirements document. 
5 The next step in the analysis stage 700 is the identification and analysis of the 

application and interface requirements, step 740. During step 740, the application and 
interface requirements are prepared based on the business and user requirements 
gathered. All agreed-upon requirements gathered to this point are entered in the 
Requirements Traceability Matrix. Step 740 is generally illustrated in FIG. 7E and 

10 comprises these steps: transforming business requirements into more detailed 

appUcation and interface requirements, step 741; integration of performance support 
requirements, step 742; recovering current application and interface design, step 743; 
identifying application and interface quality requirements, step 744; analyzing 
application and interface requirements, step 745; and verifying requirements 

15 documentation, step 746. 

During the transforming of business requirements into more detailed 
application and interface requirements in step 741, the organization uses the business 
requirements as a starting point to develop the application requirements. The 
application requirements should be in the context and scope of the business 

20 requirements. Also, these requirements should be verified to help ensure that the 

business process designs were properly interpreted. Then, in step 742, the integration 
of performance support requirements, the organization analyzes the tasks and factors 
that hinder user performance, taking into account their background and behavior. 

As illustrated in FIG. 7E, the next task in the identification and analysis of the 

25 application and interface requirements, step 740, is the recovery of current appHcation 

and interface design, step 743. The recovery of current application and interface 
design in step 743 entails reviewing the current application/interface documentation 
and physical structures to gather requirements that may be omitted from the new 
application/interface. Step 743 includes the documentation of the present logical data 

30 structures. The organization should further identify expected requirements that 

otherwise may be assumed by the business representatives and not considered. 
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Another task in step 743 is to verify that the review also covers interface 
requirements. In this way, the recovery of current appUcation/interface design in 
step 743 provides an inventory for conversion and a potential starting point for 
bottom-up data modehng. 

Subsequently, in step 744, the organization identifies application and interface 
quahty requirements, as illustrated in FIG. 7E. During step 744, the organization 
seeks to select the quality attributes used to measure the apphcation/interface 
functional and usability requirements, as these quality attributes should guide the 
design. Using these requirements, the organization should analyze application and 
interface requirements, step 745. Specifically, the organization should perform an 
analysis of the gathered requirements using process, event, data and content modehng 
techniques. Similarly, the organization may use vahdation techniques to confirm 
requirements such as prototyping and simulations. The organization may also create 
cases or scenarios to ensure requirements will be operational. The organization may 
additionally perform risk assessment against the identified requirements. The 
organization next documents the application and interface requirement specifications 
using a template. The actual requirements should be documented using a 
requirements traceability matrix for future tracking against other work products. The 
organization should make verify requirements are documented in a manner to ensure 
bidurectional traceability so that it is possible to trace requirements from the 
requirements development phase to the testing phase and vice versa. In addition, it 
should be possible to trace requirements across interfaces. In performing step 745, 
the organization preferably involves project participants impacted by the requirements 
in the review and sign-off of the requirements 

Returning to FIG. 7E, the next step is to verify the documentation of 
requirements, step 746. Specifically, the organization should review all requirement 
documents, such as executive ^chitecture, development architecture, and operational 
architecture, thereby ensuring that these documents are in sync. 

Returning to FIG. 6 A, the next set of tasks in the delivery management 600 is 
to design, step 800. The design stage 800 focuses on designing the components of the 
technology infrastructure, including the execution/operations and development 
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architectures. In addition, the design of the network, communication and computing 
platforms is performed in this stage. Design work should be coordinated with the 
development of the business processes, technical solution and organizational changes 
required to support the new infrastructure. The design process 800 is comprised of 
5 two tasks: designing the technology infrastructure, step 801 and designing the 

apphcation, step 802. 

FIG. 8 A illustrates one embodiment of the design of the technology 
infrastructure in step 801. One of the tasks in step 801 is to identify and analyze 
technology infrastructure requirements, step 810. Diiring step 810, the organization 

10 prepares for the selection and design of the technology infrastructure and establishes 

preliminary plans for technology infrastructure releases and product testing. 
Furthermore, technology-related requirements are refined to form the component 
requirements for the technology infrastructure. For instance, step 810, the 
requirements for the technology infrastructure are outlined and preliminary plans for 

1 5 technology infrastructure releases and product testing are established. As this task is 

performed, technology-related requirements are refined to form the component 
requirements for the technology infrastructure. Accordingly, a first task in the 
identification and analysis of technology infrastructure requirements during step 810 
is to identify technology infrastructure requirements, step 81 1, as illustrated in 

20 FIG. SB. The organization performs step 811 to identify the ftmctional, technical, and 

performance requirements for the technology infrastructure that should support the 
solution. During step 811, the organization also identifies key performance 
indicators, creates baseline estimates of transaction volumes and system size, and sets 
measurable targets for the performance indicators. Key performance indicators 

25 examined during step 811 include resource availability, capacity, throughput, 

reliability, scalability, and usability. 

As indicated in FIG. 8B, a second process in the identification and analysis of 
technology infirastructure requirements in step 810 is to assess the technology 
infrastructure's current environment, step 812. fri step 812, the organization assesses 

30 the ability of the existing technology infrastructure to support identified technology 

infrastructure requirements. 
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As depicted in FIG. 8B, the organization subsequently analyzes any potential 
technology infrastructure requirements, step 813, to refine the detailed functional, 
technical, and performance requirements for the technology infrastructure as outlined 
in the physical and performance models and to cover any additional requurements 
during the assessment of the current environment. The additional requirements may 
include user and service level requirements, as well as any requirements for the 
development architecture or the execution/operations architecture. The organization 
seeks to analyze and document the requirements for each component of the 
technology infrastructure and define additional needs. As part of step 813, the 
organization also seeks to involve all project participants impacted by the 
requirements in the review and sign-off of the requirements. 

Retuming to FIG. 8B, other steps in the identification and analysis of 
technology infrastructure requirements in step 810 are (1) verification that 
requirements docimientation is in sjdic, step 814, and (2) performance of risk 
assessment against the technical requirements, step 815. 

As illustrated in FIG. 8A, the next step in the design of the technology 
infi-astructure, step 801, is the selection and design of execution/operation hardware, 
step 820. The organization performs step 820 to create and document high-level 
design and component design for the execution/operation architectiire. Preferably, to 
prepare for testing of the architectural components, an architecture test plan, 
conditions, scripts and other needed family are also be created or defined during 
step 820. 

FIG. 8C depicts the individual steps of the selection and design of 
execution/operation hardware in step 820. A first step in the selection and design of 
execution/operation hardware in step 820 is to identify execution/operation 
architecture component options, step 821, so that the organization may create a list of 
suitable options for selecting and designing execution/operation architecture 
components that satisfy the technology irtfrastructure requirements. The organization 
then selects any reused execution/operation architecture components, step 822, if the 
execution architecture should utilize reused components from other projects, so that 
the organization may create a list of suitable options for selecting and designing those 
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components that satisfy the execution/operation technology infrastructure 
requirements. The organization may also select packaged execution/operation 
architecture components, step 823, if packaged components should be used in the 
project. The organization may perform step 823 to evaluate packaged products then 
and to gain the sponsoring organization's approval to continue. If suitable reusable or 
packaged components cannot be found, the organization may also choose to design 
custom execution/operation architecture components, step 824. If custom 
execution/operation components will be created in the project, the organization may 
then compare reused or packaged execution/operation solutions against custom- 
designed alternatives. 

Another step in the selection and design of execution hardware 820 is to 
design and validate the execution/operation architecture, step 825, to develop a 
complete design for the execution/operation architecture design after individual 
components have been selected or designed. The design for execution/operation 
architecture should also include custom component designs and any reused and 
packaged execution/operation architecture extension designs. 

Another step in the selection and design of execution/operation hardware 820 
is to develop an execution/operation architecture test plan, step 826, after the 
execution/operation architecture design is understood and docmnented, including the 
selection of reused and packaged execution/operation components. The primary goal 
of step 826 is to document test approaches and plans for the execution/operation 
architecture at the component and assembly level. 

The next step in the design of the technology infrastructure during step 801 is 
to select and design development architecture, step 830, as illusfrated in FIGS. 8 A and 
8D. The organization may perform this task to create and document the design of the 
development architecture components and test plans for those components. 
Specifically, the organization may create a high-level development architecture and 
component designs. Preferably, to prepare for testing of the architectural components, 
an architecture test plan, conditions, scripts and other needed family are also be 
created or defined during step 830. 
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FIG. 8D illustrates the substeps in the selection and design of development 
architecture in step 830. A first substep is to identify development architecture 
component options, step 83 1 . In step 831, the organization may create and document 
the design of the development architecture components, as well as the test plan for 
5 those development architecture components. The organization also finalizes the 
physical model and selects or designs for development architecture components. 

Other tasks in step 830 include the selection of reused development 
architecture components firom the existing technology infi-astructure or firom external 
sources, step 832, and the selection of packaged development architecture 

10 components, step 833, if they should be used in the project. If the organization should 
use any packaged development architecture components, the organization should 
determine which pieces of the development architecture to use and to negotiate 
contracts with vendors. In a preferred implementation of step 833, the organization 
also gathers additional information during vendor demonstrations and site visits to 

1 5 eval\xate the available packaged development architecture components. 

Another substep in the selection and design of development architecture of 
step 830 is to design custom development architecture components, step 834, if any 
custom-designed components are needed. The organization may choose to produce a 
design for each custom component in order to understand the complexity, effort, and 

20 skills required to design and build the components efficiently. 

In another embodiment of step 830, the organization also designs and validates 
the development architecture, step 835, to review the development architecture 
requirements such as interfaces between components, to design custom development 
architecture components, and to incorporate any reused or packaged components. 

25 The organization may also develop a development architecture test plan, step 836. 

The organization should develop a test approach and a plan for testing, concurrently 
with the design and prototyping of the development architecture. Before developing 
the test approaches and plans for each component and the assembly of the 
development architecture, the organization should further review the objectives and 

30 scope for the component, component acceptance, and assembly test approach as 

defined in the test strategy. 
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Returning FIG. 8A, a preferred embodiment of the delivery management stage 
also includes a peer review, step 840, of the other steps 810-830 undertaken during 
the process of designing the technology infrastructure, step 800. hi the peer review, 
the organization verifies the acctiracy and completeness of a deliverable product, 
whether it is a document or code, for any step in the dehvery stage 600. It should be 
appreciated that, while displayed at this point in the CMM in a BOX method 10, a 
peer review 240 may be implemented at any time as necessary to satisfy the 
requirements of the CMM or CMMI as well as other overriding business concerns. 

Referring to FIG. 8E, the organization implements the peer review by first 
preparing for the review, step 842. Specifically, the project manager and team leader 
should budget time to conduct peer reviews and to establish peer review standards and 
criteria. Also, the owner of the deUverable product should identify and contact any 
peer review participants, schedule the peer review session, and distribute materials 
and standards to the reviewers. The reviewers should then prepare for the review by 
reading the materials prior to the peer review session and documenting comments and 
recommendations. Where appropriate, a peer review checkhst may be used when 
conducting the peer review. 

Continuing with FIG. 8E, the next step in the peer review 840 is to conduct the 
peer review, step 844. During the peer review session in step 844, the dehverable 
owner should document any defects, issues, risks, and action items. The deliverable 
owner should also record meeting minutes and the time spent on the review. The 
reviewers are generally responsible for faciUtating the discussion, sharing comments 
and recommendations with the deliverable owner, confirming that all issues are 
documented, providing metrics data, and scheduling a follow-up session if necessary. 

Next, in step 846, the organization should perform any necessary rework of 
the product, as depicted in FIG. 8E, During the rework in step 846, the deliverable 
owner implements the actions recommended by the reviewers, collects metrics data 
(including time spent preparing for review, number of defects found, etc.), and 
monitors the status of defects, issues, risks, and action items. As necessary, the peer 
reviewer should also verify that all pertinent items have been closed. 
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The organization should then analyze the review results, step 848 as depicted 
in FIG. BE. The team leader submits the peer review metrics to the project manager 
for review. The project manager is then generally responsible for analyzing the 
metrics, evaluating the execution of the peer review process, and identifying areas for 
5 process improvement or corrective action with the peer review process. 

Returning to FIG. 6A, the next step in the delivery management, in step 600, is 
to design an application, step 802. As illustrated in FIG. 8F, during the design of the 
application in step 802, the organization designs an apphcation architecture, step 850, 
to develop and document the conceptual and general design of the application and 

10 designs a database, step 860, to transform the data model into logical and physical 

designs of the application's database, while ensuring that data requirements should be 
met, and that data should be available through a conversion process. The design of 
the application in step 802 also entails planning a testing approach, step 870, for 
developing a comprehensive testing approach that should be used at all levels of 

15 testing, including component, assembly, product, user acceptance testing, and 

production readiness, i.e., deployment testing. Then, in step 880, the organization 
designs a performance support approach to determine existing workforce training 
needs, as well as to design methods and standards for performance support products to 
meet those workforce training needs. 

20 During the design of the application architecture in step 850, the organization 

seeks to develop and document the conceptual, general, and interface designs of the 
application. Preferably, to prepare for testing of the architectural components, an 
architecture test plan, conditions, scripts and other needed family are also be created 
or defined. As illustrated in FIG. 8G, the first step in the design of the apphcation 

25 architecture in step 850 is to define the conceptual design, step 851. Specifically, the 

organization should document the operational concept for the solution in the 
conceptual design document. This documentation should outline the functional 
architecture of the proposed solution. 

Continuing with FIG. 8G, the organization should next determine whether to 

30 buy or build components, step 852, by reviewing the conceptual design and assessing 
factors such as historical information, corporate strategy, support infrastructure, 
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product availability, deadlines, and criticality of requirements. At this point, the 
organization shovdd define an application architecture, step 853. When defining the 
application architecture in step 853, the organization should determine an approach 
for conducting design, such as calling group meetings for creating a conceptual 
5 approach. The organization should then evolve the conceptual design into a more 

detailed design as necessary for implementation with application. While evolving the 
design, key design decisions may trigger the need for a DAR, as described above. 

Continuing with the design of the application architecture in step 850, as 
depicted in FIG. 8G, the organization next undertakes the concurrent tasks of defming 

1 0 a process flow in step 854, designing application interfaces in step 855, and planning 

an assembly test in step 856. In defming a processing flow in step 854, the 
organization identifies all programs in the application, identifies their sequence, 
decomposes the programs into modules, and identifies how the modules 
communicate. The aim of step 854 is to develop enough detail to estimate the 

1 5 application's costs, resource consumption, and response times. 

At the same time, the organization designs application interfaces, step 855. 
Specifically, the organization designs the automated interfaces between the 
application being built and other applications with which it should communicate. 
During step 855, the organization also preferably develops an interface agreement and 

20 interface design to outline the expectations of the parties developing the various 

components. These documents should describe the handling of change requests, data 
exchange and control, backup mid recovery requirements, error handling procedures, 
and provide escalation procedures in the event of a conflict. 

At the same time, the organization also plans assembly tests, step 856, by 

25 developing an approach and a plan that should be used to organize and execute 

assembly tests. The objective of assembly testing is to ensure that related components 
function properly when assembled into dialogs or batch strings and to verify that the 
component interfaces have appropriately implemented the design. 

As illustrated in FIG. 8F, the next step in the application design 802 is to 

30 design a database, step 860. When designing a database in step 860, the organization 

transforms the data model into logical and physical designs of the application's 
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database, acts to ensure that all data requirements should be met, and that all data 
should be available through the conversion process. The steps in the database design 
860 are illustrated in FIG. 8H. The first step in the database design 860 is to design a 
logical database, step 862. The organization may perform this task to transform the 
data model into the logical data structures using known database techniques. If the 
design of the logical database in step 862 produces a relational database, the logical 
database includes tables that contains various data used to define the database such as 
columns, primary keys, and row lengths; codes tables; foreign keys; integrity rules; 
views; and denormalization of the statistical data contained in the database. The 
logical data model is typically delivered to a client in soft copy format using data 
modeling tools. 

Next, the organization designs a physical database, step 864, by selecting or 
preparing physical storage and access structures for the application's data and by 
transforming the logical database design into storage and access structures that can be 
physically implemented. The physical database produced in step 864 generally 
includes database definitions, database space worksheets, database mappings, 
relational index definitions, and table space definitions. The database design 860 
continues with designing data conversion processes, step 866, such that the required 
conversion programs and procedures ensure the availability of data required by the 
application in production. In this step, the organization should produce an approach 
for converting and mapping documents. 

Returning to FIG. 8F, the design of the application in step 802 continues with 
the development of a planning testing approach, step 870. This planning testing 
approach should be used at multiple levels of testing such as component, assembly, 
product, and user acceptance testing, and deployment testing. As illustrated in 
FIG. 81, the first step in the development of a planning testing approach in step 870 is 
to develop an overall testing approach by refining and documenting an overall 
approach for testing, step 872. In developing the overall test approach, the 
organization should plan for the testing of interfaces. The overall test approach 
produced in step 872 should include details on sequence testing and the testing 
environment and also preferably includes the documentation of the resulting detailed 
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testing procediires. The next two steps in producing a testing approach in step 870 are 
(1) to identify product test conditions, step 874, where the conditions are used to 
verify that solutions meets the requirements for the components being created; and (2) 
to develop product test cycles, step 876. 
5 Returning to FIG. 8F, the next step in the appUcation design 802 is to design a 

performance support approach, step 880, to determine existing workforce training 
needs, as well as to design methods and standards for performance support products to 
meet these workforce training needs. In step 880, the organization also designs 
performance support test and evaluation approaches and completes a validation of the 

10 complete test and evaluation approach. With reference to FIG. 8 J, the first step in the 

design of a performance support approach is to determine performance support needs, 
step 881, to determine the workforce's current proficiency and performance levels. 
This information is used to assess the gaps between current and expected proficiency 
and performance levels, which, in turn, drive the design of the performance support 

15 approach. Next, the organization designs learning objectives and a curriculum plan 

necessary to close the proficiency and performance gaps in the organization's 
workforce, step 882. Another step of the design of a performance support approach is 
to design performance support products, step 883, to define the delivery methods and 
standards for performance support. These delivery methods may include instructor- 

20 led training, performance simulation, computer-based training, videos, workshops, job 

aids, on-line quick reference tools, and training databases. 

As illustrated in FIG. 8J, the next step is to design a comprehensive approach 
for testing the performance support products with respect to achieving each product's 
learning objectives, step 884. In step 884, the organization generally defines an 

25 approach that includes the scope and objectives of the test, environment requirements, 

entry/exit criteria, metrics, and schedule. The organization then designs a 
comprehensive approach for evaluating the effect of the performance support products 
on the employees' competency proficiency levels and performance levels in specific 
areas, step 885. Any designed approach for performance support evaluation should 

30 include evaluation methods, proficiency metrics, and schedules. The design of the 

performance support approach in step 880 may also include the verification and 
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validation of the performance support approach and the curriculum plan with 
stakeholders and subject matter experts, step 886. The organization should also 
organize labor review sessions to determine how well the sessions fit together to 
support the training needs of the workforce. 
5 As illustrated in FIG. 6A the next step in the delivery management 600 is to 

build and test, step 900. The build and test step 900 concentrates on implementing the 
business solution elements required for a single release. The deUvering teams are 
responsible for the detailed design and creation of new processes, facilities, learning 
systems, performance support, application systems and technology infrastructure 

10 necessary to implement the new solution. These elements are then tested and 

implemented within a pilot environment. Thus, the building and testing in step 900 is 
accomplished through building and testing the technology infrastructure in step 901, 
building and testing the apphcation in step 902, and plaiming executing product and 
acceptance tests in step 903. 

15 FIG. 9A presents the elements in the building and testing of the technology 

inj&astnicture in step 901. Step 901 focuses on acquiring, developing and testing the 
technology infrastructure. During step 901, additions and extensions to the 
execution/operations and development architectures are implemented, physical 
network and computing resources are developed, and a imified product is tested prior 

20 to the application product test. The first task in step 901 is to acquire physical 

enviroimient assets and services, step 905. Generally, these physical environment 
assets and services are deployed to enable the implementation of the requirements 
based on the previously defined details of the physical environment assets. For 
instance, the organization may apply the data obtained in step 420. The organization 

25 uses the listing of physical enviroimient assets and services to decide who should 

supply the assets and services, how the assets and services should be supplied, and 
how much the assets and services may cost. 

As depicted in FIG. 9B, the first step for acquiring physical enviroimient 
assets and services is to initiate the acquisition of physical environment assets and 

30 services by selecting and appointing providers of assets and services, step 906. For 

instance, the organization preferably identifies those contracts that need to be 
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negotiated on an expedited basis and ensures that due diligence is applied to the 
context and content of all contractual arrangements. The organization then selects 
and appoints assets and services vendors, step 907, to appoint third-party supphers 
and contractors who may provide assets, such as property and equipment, and 
technical/build/transfer/install/maintenance services for deployment of the physical 
environment, or services relating to decommissioning and disposal of the existing 
physical environment. 

Again, the organizations should prioritize those early purchase requirements 
that need to be expedited on a "fast track" basis. Subsequently, the organization 
should evaluate the deployment implications of the vendor appointments, step 908, to 
analyze the impact and deployment implications of appointing specific providers, 
either external or internal. These impacts may involve additions or revisions to project 
documents such as deployment plans, Business Case, project plan, and all subordinate 
plans. 

Returning to FIG. 9A, the next step in the building and testing of the 
technology infrastructure, step 901, is to build and test the execution/operation 
architecture, step 910, in order to complete a detailed design of the 
execution/operation architecture and to build and test that architecture. The 
organization may use the same methodology for application and operation 
development, as provided above in step 820, to plan and perform the component tests 
of the execution/operation architecture. As illustrated in FIG. 9C, the first step in the 
building and testing of the execution/operation architecture is to develop program 
specifications for each custom component of the execution architecture and to 
determine software configvirations for each packaged or reusable component of that 
architecture, step 911. The organization may use the resulting detailed design to build 
custom components and to install packaged or reusable components. This task may 
also include updating the technology infrastructure component test plans, conducting 
reviews of the resulting detailed designs, and preparing common test data. 

The organization next builds any custom execution/operation architecture 
components needed for the project, step 912. This step 912 may also include 
docimienting development procedures and standards, and conducting code reviews. 
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The organization then prepares and executes a component test of the 
execution/operation architecture components, step 913, to verify that the 
execution/operation architecture components are built according to proper designs. 
During step 913, any detected errors should be documented, said all of the execution 
5 architecture components should be relatively free of errors and ready for a subsequent 

assembly test. 

As depicted in FIG. 9D, the organization should similarly build and test the 
development architecture, step 915. The first task in step 915 is to perform a detailed 
design of the development architecture, step 916. In step 916, the organization 

10 develops program specifications for each custom component of the development 

architecture and to determine software configurations for each packaged or reused 
component of that architecture. Step 916 also preferably includes updating the 
technology infrastructure component test plans, conducting reviews of the resulting 
detailed designs, and preparing common test data. The organization should then bmld 

15 any needed custom development architecture components, step 917. Step 917 may 

also include docvimenting development procedures and standards, and conducting 
code reviews. The organization then prepares and executes a component test of the 
development architecture components, step 918, to verify that the development 
architecture custom components are built according to their designs. During step 918, 

20 the detected errors should be documented, and all of the development architecture 

custom components should be relatively free of errors and ready for the assembly test. 

As depicted in FIG. 6A, the build and test stage 900 also includes the building 
and testing of the application in step 902. Step 902 focuses on building and testing 
the application, creating training materials and other forms of performance support 

25 required by the business solution. During step 902, the detailed design, component 

testing and assembly testing of the application are completed. Learning products and 
business poUcies and procedm-es are developed to train and guide the users of the 
application. FIG. 9E illustrates tiie steps involved in the process to build and test the 
application, step 902. The first of these tasks is deployment planning, step 930, to 

30 produce deliverables that should be needed to test the application and interfaces in an 
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operations environment prior to deployment and to run the application and interfaces 
after deployment has occurred. 

Turning to FIG. 9F, the first task in the deployment planning during step 930 
is to develop a deployment approach to document the specifics of the major 
5 deployment activities, step 931. The documentation should include items such as 

Data Conversion, Policy & Procedure Deployment, Risk Mitigation, Deployment 
Strategy, and workforce transition also should be covered in this document. The 
organization should next develop appropriate operating pohcies to produce a 
document outlining specific policies in the new operation environment, step 932. 

10 Responsibilities, system availability, and security should be documented in step 932, 
and upon completion of the project, this documentation should be given to the client. 
In a concurrent task, step 933, the organization may develop operating procedures by 
producing a document outlining the procedures that need to be followed during on- 
going support and operation of the installed application. Other subsequent tasks are to 

15 develop the operating organization to document the long-term organizational 

requirements that should be needed in the new operation environment, step 934, and 
to develop a disaster recovery plan that outlines an overall disaster recovery approach, 
as well as specific steps to follow during the disaster recovery process, step 935. The 
organization may then prepare the deployment test by creating a deployment test plan, 

20 test conditions, test scripts, and test data, step 936. This plan should be executed prior 
to delivery of a product to clients. Another step is to package operating manuals, so 
that the manuals may be turned over to client at completion of the project, step 937. 

The next step 940, the performing of application detailed design, is illustrated 
in FIG. 9G, and generally comprises a process to produce completed detailed design 

25 specifications that can be directly implemented in code, and a process to develop the 

approach and plan for component testing the appUcation's modules. The first 
substep is to design and specify modules, step 941 . Step 941 includes the production 
of a detailed design of the apphcation and interfaces based on the general design and 
the application/interface requirements specification. During step 941, the 

30 organization should continue use of the chosen design methods to complete detailed 

designs. The organization should also prepare a detailed design of the application by 
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specifying all of the modules and their associated call patterns to the lowest level of 
detail. The detailed design of the apphcation should also include describing each 
module's purpose and processing logic, developing database access patterns, and 
identifying other input/output operations. The organization should also be sure to 
address interfaces during the design process. In step 941, the organization should also 
update the interface agreement created during the design stage 600 to reflect any 
changes associated with the interfaces. The next task in performing a detailed design 
of the application in step 940 is to plan component testing, step 942, to verify the 
correctness of implementation of each of the application modules with respect to the 
application detailed design specifications. Step 942 includes determining conmion 
test data requirements and using the requirements to create common test data that can 
be used in the different stages of testing. 

As illustrated in FIG. 9H, the next step in the build and test stage 900 is to 
build and test the application, step 945. In step 945, the organization builds a 
complete, high-quality software application from the detailed design of the 
application. The organization may have developers implement the modiiles and then 
review the coded modules to verify correctness. The organization may also execute 
assembly tests to check interfaces and interdependencies between modules. One task 
in the building and testing of the application is the coding of modules, step 946, to 
create the code of each of the modules according to the previously created detailed 
application design specifications. Once the code is generated, the organization should 
check and compile the code as necessary for the project to identify and fix all errors, 
and to ensure that developers have followed any detailed coding procedures outlined 
in the project developer's guide. 

Continuing with FIG. 9H, the goal of the next step, the preparation and 
execution of the component tests in step 947, is to execute module code and verify 
that the module specification was correctly translated to the code. The module code 
should be verified using the component test conditions from the component test plan 
to prepare the test data and test scripts for the component tests. The organization 
should document and fix all detected errors before proceeding. The organization may 
then prepare and execute assembly tests, step 948, as needed to integrate modules and 
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verify that their interfaces and interdependencies are correctly designed and 
implemented. In step 948, the organization should use the assembly test conditions 
from the previously prepared assembly test plan to prepare test data and test scripts 
for the assembly tests. All detected errors should be fixed before proceeding. The 
5 next step, the development of a support program in step 949, involves coordinating 

and controlling the efforts of the development teams by supporting the programming 
and testing effort through supervision, control, and coordination. The organization 
may manage the programming and testing schedule, and monitor progress and report 
status, via the project management task packages outlined in the document repository 

1 0 policy defined in earlier steps. 

As depicted in FIG. 91, at this point in the build and test stage 900, the 
organization may develop a finalized, detailed set of policies and procedures, 
step 950. The business policies and procedures consist of rules governing work 
within the organization (policies) and the workflow for executing these rules 

1 5 (procedures). A first task in step 950 is to perform a detailed design of pohcies and 

procedures, step 952. In step 952, the organization should (1) define the product 
structure and design and (2) create and develop prototype templates for all pohcies 
and procedures. The organization should then develop business policies and 
procedures, step 954, by drafting a complete set of business policies and procedures to 

20 support the pending product release. In step 954, the business pohcies describe the 

business rules governing workflows and drive the development of business 
procedures and user procedures documentation. Similarly, the business procedures 
describe the sequential sets of tasks (and related resources, metrics, etc.) to follow 
based on the business policies. The organization should next validate and test these 

25 policies and procediu-es, step 956, to ensure that the Business Policies and Procedures 

meet the content of the requirements and can be executed by use of the applicable 
application. In step 956, the organization should further verify that the information 
collected is complete and accurately describes the processes. 

Turning to FIG. 9J, the next task in the building and testing of an appUcation 

30 in step 900 is to develop learning products, step 960. In step 960, the organization 

selects the relevant authoring and development tools and to define standards, 
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templates, and development procedures. Step 960 further includes the defining of 
detailed learning objectives, determining learning context, and designing learning 
activities. The organization should also review paper-based learning product 
prototypes for ease of use. Also, the organization should develop activities and 
5 content, and define the support learners should require, and develop learning program 

evaluation materials for during-delivery and post-delivery evaluation of the learning 
process. Thus, in step 960, the organization should prepare said execute testing to 
ensure each learning product meets the stated objectives and instructors are effective 
when using the learning products. 

10 As depicted in FIG. 9 J, one task in the development of learning products in 

step 960 is to define learning product standards and a development environment, 
step 961, after the scope of the learning program has been defined and the learning 
requirements have been identified. In step 961, the organization should further select 
authoring and development tools and define the standards, templates, and procedures 

1 5 for the learning products. Development environments typically include Word or 

PowerPoint-based instructor-led materials or computer-based applications, but can 
also be made more robust with the use of job aids, a training database, on-line quick 
reference tools, and videos. 

Returning to FIG. 9J, concurrent steps in the developing of learning products 

20 in step 960 are (1) performing a detailed design of a learning program, step 962, to 

specify how each learning product identified in the learning product design should be 
built to meet the business needs of the organization and (2) prototyping learning 
products, step 963, to complete low-fidelity prototypes and conduct ease-of-use 
sessions on learning components (e.g., activities, support system, and instructor 

25 guide) of classroom-based learning products. 

In addition, the organization may create learning and evaluation products, step 
964, to develop the learning materials proposed and prototyped during the learning 
design activities. The creation of learning and evaluation products in step 964 
involves the developing of activities, content, and support materials that the learner 

30 will require to complete the learning product. Furthermore, evaluation tools are also 

preferably created in step 964 to ensure that learners have met the learning objectives. 
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Another possible task in the development of learning products in step 960 is the 
testing of learning products, step 965, which is best implemented after documenting 
participant profile, sample size, learning testing methods, test cycles, expected results, 
and script outlines. The goal is to test each learning product with the intended 
5 audience to ensure that the product meets the stated learning objectives, that the 
instructors are effective, and that the learning product meets the overall learning 
objectives for the release. The organization may also package the learning products, 
step 966, so that the learning products may be handed over to an appropriate 
stakeholder at the end of the project. 

10 At this point, the organization may plan and execute the product test and 

acceptance test, step 903, as illustrated in FIGS. 6 and 9K. Product tests evaluate 
whether the product is properly fimctioning, whereas acceptance tests evaluate 
whether the product functions as desired by customers. Step 903 focuses on 
performing a product test and user acceptance test on the new application to verify the 

1 5 application components and related technology, processes, and procedures work 

together properly according to the application and interface requirements. The first 
task in step 903 is to prepare and execute a product test plan, step 970, following the 
creation of the product test plan, conditions, scripts, and data that are used to execute 
the product test. The planning and execution of the product test plan in step 970 

20 should not begin until all requirements are finalized, the assembly test has been 

successfully completed, and the testing approach has been finished. 

As illustrated in FIG. 9L, to prepare and execute a product test, the 
organization should prepare a product test plan, step 971, to design and create the test 
conditions, test scripts, and test data for product testing. The organization should then 

25 review its product test plan, step 972, to verify that the product test plan created in 

step 971 is complete and accurate prior to product test execution. The resulting 
benefit to this check is that errors are caught early in the test process, where they can 
be addressed with minimal effort, rather than during test execution, where correction 
of errors becomes more costly. The organization should also create, cleanse, and 

30 convert data, step 973, to prepare the data for product test execution. If needed, the 

organization may confirm the product test environment, step 974, to verify that the 
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product test environment is ready for application product test execution by confirming 
that associated items are transferred to the test environment and that the identified 
configuration is complete and accurate. In this way, in step 974, the organization 
verifies that any tools needed for managing and executing the product test (for 
5 example, scripting tools and test data management tools) are installed and fliUy 

operational. This step 974 also helps ensure that the test data is properly copied and 
identifies responsibility and authority levels for managing code migration into the 
product test enviroiraient. 

Continuing with FIG. 9L, following confirmation of the test, the organization 

10 may execute the product test, step 975, to verify that the new application can work 

with the related technology, processes, and procedures to support the business 
processes successfiiUy. The product test should prove; (1) that the new apphcation 
and interfaces perform according to the application/interface requirements established 
in prior steps, and (2) that the application can operate effectively in concert with all 

1 5 other production applications and all available end-user manuals and procedures. 

If any problems arise during the testing, the organization may perform product 
test fixes, step 976, to analyze and resolve all problems identified during product test 
execution as illustrated in FIG. 9L. Typically, the organization assigns each problem 
to a specific team member for correction. After a problem is fixed, the organization 

20 may reexecute the test condition to verify that the fix was successfiil, and perform a 

regression test to ensure other components were not adversely affected by the fix. 
Once all errors have been resolved the product test can be considered complete. 

Returning to FIG. 9K, the organization may next prepare and execute 
acceptance tests, step 980. The organization performs step 980 to create the test plan, 

25 test conditions, test scripts, and test data for user acceptance testing. The user 

acceptance test (UAT) also vahdates that the solution supports the business 
performance model and should not begin until successfiil completion of the product 
test in step 970. The UAT verifies that the solution works according to the 
requirements and meets the business objectives. As depicted in FIG. 9M, steps 981- 

30 986, the preparation and execution of the acceptance tests during step 980 are very 

similar to steps 971-976. For instance, the initial step of the preparation and 
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execution of the acceptance tests in step 980 is to prepare a user acceptance test plan, 
step 981, including plans for testing interfaces and the application. The organization 
then reviews the user acceptance test plan, step 982, to ensure that the user acceptance 
test plan is complete. The next step is to create, cleanse, and convert data, step 983, 
5 as needed, to prepare the data required for the acceptance testing, including producing 

new data, converting existing data, and reconciling different data representations and 
different database schema representations. If necessary, the organizations may also 
confirm user acceptance of the test environment, step 984, to ensure: (1) that the user 
acceptance test environment is ready for test execution by checking that all necessary 

1 0 items are transferred to the test environment, (2) that the identified configuration is 
complete and accurate, and (3) that any tools required during the acceptance test are 
installed and fully operational. 

At this point the organization executes the user acceptance test, step 985, to 
test the interaction between the components of the solution to verify and validate that 

1 5 they support the model. This acceptance test helps to ensure that the solution works 
according to the requirements and meets the business objectives. If any problems 
arise in the test, the organization may resolve user acceptance test issues, step 986. 
Specifically, the organization may utilize the user acceptance test issues to analyze all 
problems identified by the user acceptance test execution through investigating each 

20 problem, and assigning it to the appropriate development team for correction. After a 

problem is fixed, the organization should reexecute the test condition to verify the fix 
was successful. The organization may also perform a regression test to ensure other 
components were not adversely affected by the fix. Once all errors have been 
resolved in step 986, the acceptance test may be considered complete. 

25 Once solutions to a problem have been analyzed in step 700, designed in 

step 800, and built and tested in step 900, an organization may deploy the complete 
solution, as depicted in FIG. lOA. The deployment stage 1000 is conducted to 
transition the organization to the new business solution. The deployment stage 1000 
includes the activities required to transform the personnel, business process, and 

30 technology elements required to estabhsh the business solution. The deployment 

stage is repeated for each deployment site, which is the organizational or geographic 
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unit that will receive the business solution. The first step in the deployment is to 
transition users and to deploy policies and procedures, step 1010, to evaluate the 
existing workforce of an organization in terms of roles and skills, and perform a gap 
analysis against the new organization infrastructure for the deployment unit, as 
5 illustrated in FIG. lOB. In step 1010, the organization may finalize the workforce 

infrastructure, step 101 1, to mobilize the people who should eventually use the 
solution. At the same time, the organization should examine the organizational 
structure, as well as the skills and roles of the existing workforce, to determine if the 
resources needed to support the solution exist. If needed roles or skills are missing, 

10 another task in step 101 1 is to develop a plan to address the gaps. This task should be 
performed before selecting, hiring, or assigning people to teams. 

As illustrated in FIG. 1 OB, the next task is to redeploy the workforce, 
step 1012, to transfer existing users into the different roles, teams, or fimctional areas 
needed to support the solution. Concurrently, the organization recruits and selects a 

15 workforce, step 1013, after developing a profile of the combination of skills and other 

characteristics necessary to support the solution and using the resulting profile to 
select internal individuals and to hire external individuals who can fill the necessary 
roles and teams. The organization then trains the trainers, step 1014, by preparing the 
instructors and coaches who should eventually train the workforce to use the solution. 

20 Step 1014 generally entails conducting practice sessions of the course in order to 

allow instructors to rehearse their delivery with course developers as the audience. 
Next, the organization implements orientation and training, step 1015. Specifically, 
the organization introduces employees to the solution that should be deployed. To 
maximize the benefits of training in step 1015, the instructors should be trained in 

25 step 1014 prior to the training of the workforce. The organization may further give 

users information on the context of the solution within the organization and frain them 
on how to operate the solution. Furthermore, the organization preferably identifies 
individual and team development needs, and workers should provide feedback on the 
learning program in order to improve the process for fixture releases. Step 1015 

30 should be performed after selecting and recruiting individuals to fill the roles and 

teams, and afi;er developing the fraining materials and job aids. 
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Concurrent with above-described steps 1011-1015, if needed in response to 
the deployment, the organization may install the new business policies and 
procedures, step 1016. In step 1016, the organization also acts to en sure that all 
pieces of the new business policies and procedures are available. 
5 The next step of deployment stage 1000 is to deploy the physical environment, 

step 1020, as illustrated in FIG. IOC. In step 1020, the organization manages the 
implementation of changes to facilities, equipment, and other physical assets. Upon 
completion of step 1020, a formal exchange of the transformed physical environment 
from the project team to the sponsor's operating management may occur. 

1 0 Continuing with FIG. 1 OC, one task in deploying physical environment in step 

1020 is to initiate physical environment deployment, step 1022, to mobilize the 
internal and external resources to prepare the physical environment for the solution 
that should be deployed, and to establish the necessary commxmication channels. One 
aspect of step 1022 is to verify that all of the involved parties understand what work 

1 5 for which they are responsible, when this work is scheduled, and how this work is 

interdependent with the tasks assigned to others. Other tasks in step 1022 may include 
defining how to monitor, expedite, and report progress. The organization may 
optionally determine how to maintain quality control and how to regularly 
communicate progress with stakeholders. Also, step 1022 may include planning for 

20 formal progress and quaUty control reviews. 

Returning to FIG. IOC, the next step in the deployment of the physical 
environment during step 1020 to is to manage physical environment transformation, 
step 1024, to carry out the development and configuration of the physical 
environment needed to support the solution. The management of physical 

25 environment transformation in step 1024 includes expediting progress, managing 

issues and risks that may impact the implementation plan, and providing management 
with summary progress reports. 

Continuing with FIG. IOC, another the next step in the deployment of the 
physical environment during step 1020 is to complete a physical environment 

30 handover, step 1026. During 1026, the organization acts to ensure that the 

development and configuration of the physical environments are complete, and are 
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transferred to, and accepted by, the sponsoring organization's operations 
management. Step 1026 generally occurs when both the stakeholders and the 
deployment project management team are satisfied that the implementation has been 
completed successfiilly. 

As depicted in FIG. lOD, the next task is to deploy the appHcation, step 1030, 
to transition the new application and its operating environment into the deployment 
unit. Dijring step 1030, the organization may establish the data required by the new 
application; configure the operating environment to the needs of the deployment unit; 
install the application; configure application parameters needed for the deployment 
iuiit; and verify that the application is correct and consistent for the deployment. 
Tasks in step 1030 may include the creation, cleansing, and conversion of data, 
step 1032, as needed, to establish the data to be used with the new apphcation. 
During step 1032, an organization may produce new data and reconcile different data 
representations and different database schema representations. The organization may 
also convert an existing electronic representation of data into a format to be used by 
the new application or use a data conversion application to convert data fi-om an 
existing database to the new database. 

As depicted in FIG. lOD, a concurrent task is to configure the application, step 
1034 in order to configure and customize the new application and the existing 
operating environment to the needs of the deployment unit. Next, the organization 
installs the application, step 1036. Specifically, the organization may, during step 
1036, install and customize the application components of the business capability in 
the deployment unit, making sure that all pieces of the new application are available. 
Another task in the deployment of the application during step 1030 is to verify 
application, step 1038, by installing and customizing the new application components 
of the business capability in the deployment unit, making sure that all pieces of the 
new application are available. 

As illustrated in FIG. lOE, another step in the deployment stage 1000 is to 
deploy the technology infrastructure, step 1040. During step 1040, the organization 
preferably outlines of the procedures and considerations for deploying technology 
infrastructure components at a deployment unit. Likewise, the organization should 



-89- 



Atty. Dckt. No.: 86769-006 



address the potential differences in technology infrastructure environments between 
deployment units. The goal of step 1040 is to bring the deployment unit up to the 
technology infrastructure baseline required for the business capabiUty. Deployment 
of the technology infrastructure in step 1040 may also include the commissioning and 
decommissioning of infrastructure components. To deploy the technology 
infrastructure in step 1040, the organization may also configure the technology 
infrastructure, step 1042, to customize the deployment unit's technology 
infrastructure in preparation for the new business capability components. Step 1042 
generally does not handle the configurations that are part of the installation of any 
new technology infrastructure elements. Next, the organization installs the 
technology infrastructure, step 1044, to install the technology infrastructure 
components of the business capability. The organization should also verify the 
available technology infrastructure, step 1046, so that whenever a technology 
infrastructure component is added or modified, the organization performs this task to 
verify the new technology infrastructure environment and addresses the discoveries of 
the testing. This verification in step 1046 is generally completed only for the 
technology infrastructure. 

The next task in the deployment stage 1000 is to activate and test a solution, 
step 1050, to verify the deployment and launch the new operating management 
processes. Step 1050 generally includes actions required to finalize performance 
targets, to remove redundant legacy elements, and to stabilize the deployment unit for 
transition to operations management. One task in the step 1050 is to verify workforce 
and business readiness, step 1051, after successftil completion of the acceptance test 
and after all elements have been deployed, but before the business capabiUty is 
activated. Step 1051 includes execution of the deployment test and verifies that the 
workforce and the other elements of the business are prepared for operation on the 
first day and all subsequent days. The organization may use the SIRs or CRs to 
record any errors encountered. 

A concurrent task is to verify team and process readiness, step 1052, after all 
elements have been deployed, but before the business capability is activated. Step 
1052 verifies that the deployment team and the deployment processes are prepared to 
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activate the new business capability. Organizations may also activate and verify the 
deploj'ment, step 1053, to activate and verify the capabilities that have been deployed. 
In step 1053, any of the organization's various teams should have the confidence and 
ability needed to proceed with irreversible decisions, such as the removal of legacy 
5 systems and procedures. The organization should now begin to operate the deployed 

business capabilities. 

Next, the organization may remove legacy elements, step 1054, to remove the 
legacy systems from old operations and management processes after making the 
irreversible decision to proceed with the new business solution. Concurrent with step 

10 1054, the organization should finalize performance targets, step 1055 to formalize the 

baseline for continuous improvement of the business solution. The finalizing of 
performance targets is initiated as soon as the business solution has been operating 
long enough to collect reliable data for adjusting the business performance model. 
In another step, the organization may deploy stabihzation, step 1056, to 

15 prepare the transition of business capabilities to operations management. The 

organization should also monitor the progress over a period of time to verify the 
stabiUty of the team using the deployed business capabilities. A decision that the 
product is ready to release is reached by analyzing the actual performance and 
productivity forecasts of the team using the deployed business capabilities. 

20 Tuming now to FIG. 6B, maintenance, step 610, is the continuing support of 

an application, addressing both production problem resolution (through SIRs) and 
application enhancements (through CRs). The first task in the maintenance is to 
review the SIRs or CRs, step 611. With a SIR, repair work needs to be completed 
immediately, whereas a CR may be incorporated into a subsequent release of the 

25 application. The organization may also review incident or change requests for risk as 

well. Another step in the maintenance 610 is to perform an impact assessment, 
step 612. Specific activities in step 612 include investigating the SIR; determining the 
change required to address the identified problems to resolve the SIR; determining the 
effort involved; developing alternatives; and selecting the acceptable alternative. Any 

30 affected work products altered by the SIR such as requirements, designs, work plans, 

code, etc., should be updated as necessary. If it is determined that no application 
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change is needed, the system should be retested to ensure that the problem no longer 
exists or that the problem should be forwarded to the appropriate channels. 

Another task in the maintenance 610 is to design application changes, step 
613, to create the application design that is needed to build the solution. The 
5 organization may also build and test application changes, step 614, to perform the 

work necessary to implement the desired change. Once the change has been 
completed, the change should be component tested and product tested to ensure that it 
is working properly. Additionally, a regression test should be performed in step 614 
to help ensure that other peripheral fimctions were not affected by the change. Next, 

10 the organization may roll out changes, step 615, as needed to implement the designed, 

developed, tested changes into the production environment. 

For CRs corresponding with desired enhancements to the product, the 
organization may also follow the program delivery life cycle, step 616: For changes 
(CRs) that can be incorporated into a scheduled release, the detailed work involved in 

1 5 modifying the existing application is performed according to the task packages/tasks 

in the delivering phase 600, including the analysis, design, build and test, and 
deployment steps 700, 800, 900 and 1000. In this way enhancement that extend 
beyond the original scope of the product are developed much Uke a new product. 

20 Svstem 

Those skilled in the art of process engineering will recognize that various 
embodiments of the CMM in a BOX method 10 described above may be implemented 
in various ways. For instance, the organization may use a set of written templates 
directing the implementation of the tasks in the CMM in a BOX method 10. 

25 hi one implementation, the present invention may be implemented as a 

computer apphcation that prompts an organization for various inputs regarding its 
operation and structure. Using these inputs, the apphcation then creates a series of 
task lists to implement the CMM in a BOX method 10 of the present invention. The 
application may further create a record of task lists, so that the organization may 

30 easily document its actions as required in the CMM and CMMI. Alternatively, the 
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program may provide templates through which the organization may docviment its 
activities. 

In particular, those skilled in the art will recognize that various embodiments 
of the CMM in a BOX method 10 described above may be implemented using a 
combination of both electronic hardware and software. Referring to Fig. 1 1 A, a 
CMM implementation system 1 100 receives user input 1 130 and produces a business 
organization plan 1 140 based on the user input 1 130. The system 1 100 may be, for 
example, a personal computer (PC), a server, or any other computer device used for 
such purposes. The system 1 100 may be coupled to a database 1 120 containing 
information on the organization and its suppliers. In this embodiment, the system 
11 00 has an organization management module 1 1 10, a program management module 
1 112, a project management module 1114 and a dehvery management module 1116 
for implementing organization management 100, program management 400, project 
management 500, and delivery management 600. 

If the computer device 1100 is, for example, a network server, in electronic 
communication with an electronic network, then users 1 160 may be able to use the 
CMM system 1 100 remotely. Referring to Fig. 1 IB showing the computer device of 
Fig. 1 1 A in electronic communication with a network 1 150. The network 1 150, may 
be, for example, the Internet, an intranet, an extranet, a Value Added Network 
("VAN"), Virtual Product Network (VPN) and the like. Users 1 160 may transmit user 
input data 1 120 to the CMM system 1 100 via the electronic network 1 150 then obtain 
a business organization plan 1 140 based on the input data 1 130. 

In another embodiment, the CMM system 1 100 illustrated in FIGS. 1 lA-B, 
may be a software application designed to operate over various hardware and 
computer systems, as known in the art. 

Conclusion 

The CMM method of the present invention has been empirically shown to 
allow organizations to achieve higher levels of CMM hierarchy much more rapidly. 
On average, an organization or a project within an organization takes about three 
years to achieve compliance with level 3 of the CMM. In contrast, several projects 
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implementing the CMM in a BOX method 10 of the present invention have reached 
level 3 of the CMM in an average of nine months. These results suggest the utihty and 
benefit of the present invention in assisting organizations to achieve higher levels of 
CMM maturity. 

The foregoing description of the preferred embodiments of the invention has 
been presented for the purposes of illustration and description. It is not intended to be 
exhaustive or to limit the invention to the precise form disclosed. Many modifications 
and variations are possible in light of the above teaching. For mstance, the method of 
the present mvention may be modified as needed to meet the requirements of new 
versions of CMM and other maturity models as they are developed. It is intended that 
the scope of the invention be limited not by this detailed description, but rather by the 
claims appended hereto. The above specification, examples, and data provide a 
complete description of the manufacture and use of the composition of the invention. 
Since many embodiments of the invention can be made without departing fi-om the 
spirit and scope of the invention, tbe invention resides in the claims hereinafter 
appended. 
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